KVM Archive on lore.kernel.org
 help / color / Atom feed
From: Nitesh Narayan Lal <nitesh@redhat.com>
To: Alexander Duyck <alexander.duyck@gmail.com>
Cc: kvm list <kvm@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	linux-mm <linux-mm@kvack.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	lcapitulino@redhat.com, Pankaj Gupta <pagupta@redhat.com>,
	"Wang, Wei W" <wei.w.wang@intel.com>,
	Yang Zhang <yang.zhang.wz@gmail.com>,
	Rik van Riel <riel@surriel.com>,
	David Hildenbrand <david@redhat.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	dodgen@google.com, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	dhildenb@redhat.com, Andrea Arcangeli <aarcange@redhat.com>,
	john.starks@microsoft.com, Dave Hansen <dave.hansen@intel.com>,
	Michal Hocko <mhocko@suse.com>,
Subject: Re: [RFC][Patch v12 1/2] mm: page_reporting: core infrastructure
Date: Fri, 16 Aug 2019 14:35:50 -0400
Message-ID: <cc681772-40dd-d429-1715-96498fa9c48e@redhat.com> (raw)
In-Reply-To: <CAKgT0UduKXTMHD2qWqEa7wQPOFYtaQ5Sx3XS9Ki8i8-_kTdmkg@mail.gmail.com>

On 8/15/19 7:00 PM, Alexander Duyck wrote:
> On Thu, Aug 15, 2019 at 12:23 PM Nitesh Narayan Lal <nitesh@redhat.com> wrote:
>>>>>>> +}
>>>>>>> +
>>>>>>> +/**
>>>>>>> + * __page_reporting_enqueue - tracks the freed page in the respective zone's
>>>>>>> + * bitmap and enqueues a new page reporting job to the workqueue if possible.
>>>>>>> + */
>>>>>>> +void __page_reporting_enqueue(struct page *page)
>>>>>>> +{
>>>>>>> +       struct page_reporting_config *phconf;
>>>>>>> +       struct zone *zone;
>>>>>>> +
>>>>>>> +       rcu_read_lock();
>>>>>>> +       /*
>>>>>>> +        * We should not process this page if either page reporting is not
>>>>>>> +        * yet completely enabled or it has been disabled by the backend.
>>>>>>> +        */
>>>>>>> +       phconf = rcu_dereference(page_reporting_conf);
>>>>>>> +       if (!phconf)
>>>>>>> +               return;
>>>>>>> +
>>>>>>> +       zone = page_zone(page);
>>>>>>> +       bitmap_set_bit(page, zone);
>>>>>>> +
>>>>>>> +       /*
>>>>>>> +        * We should not enqueue a job if a previously enqueued reporting work
>>>>>>> +        * is in progress or we don't have enough free pages in the zone.
>>>>>>> +        */
>>>>>>> +       if (atomic_read(&zone->free_pages) >= phconf->max_pages &&
>>>>>>> +           !atomic_cmpxchg(&phconf->refcnt, 0, 1))
>>>>>> This doesn't make any sense to me. Why are you only incrementing the
>>>>>> refcount if it is zero? Combining this with the assignment above, this
>>>>>> isn't really a refcnt. It is just an oversized bitflag.
>>>>> The intent for having an extra variable was to ensure that at a time only one
>>>>> reporting job is enqueued. I do agree that for that purpose I really don't need
>>>>> a reference counter and I should have used something like bool
>>>>> 'page_hinting_active'. But with bool, I think there could be a possible chance
>>>>> of race. Maybe I should rename this variable and keep it as atomic.
>>>>> Any thoughts?
>>>> You could just use a bitflag to achieve what you are doing here. That
>>>> is the primary use case for many of the test_and_set_bit type
>>>> operations. However one issue with doing it as a bitflag is that you
>>>> have no way of telling that you took care of all requesters.
>>> I think you are right, I might end up missing on certain reporting
>>> opportunities in some special cases. Specifically when the pages which are
>>> part of this new reporting request belongs to a section of the bitmap which
>>> has already been scanned. Although, I have failed to reproduce this kind of
>>> situation in an actual setup.
>>>>  That is
>>>> where having an actual reference count comes in handy as you know
>>>> exactly how many zones are requesting to be reported on.
>>> True.
>>>>>> Also I am pretty sure this results in the opportunity to miss pages
>>>>>> because there is nothing to prevent you from possibly missing a ton of
>>>>>> pages you could hint on if a large number of pages are pushed out all
>>>>>> at once and then the system goes idle in terms of memory allocation
>>>>>> and freeing.
>>>>> I was looking at how you are enqueuing/processing reporting jobs for each zone.
>>>>> I am wondering if I should also consider something on similar lines as having
>>>>> that I might be able to address the concern which you have raised above. But it
>>>>> would also mean that I have to add an additional flag in the zone_flags. :)
>>>> You could do it either in the zone or outside the zone as yet another
>>>> bitmap. I decided to put the flags inside the zone because there was a
>>>> number of free bits there and it should be faster since we were
>>>> already using the zone structure.
>>> There are two possibilities which could happen while I am reporting:
>>> 1. Another request might come in for a different zone.
>>> 2. Another request could come in for the same zone and the pages belong to a
>>>     section of the bitmap which has already been scanned.
>>> Having a per zone flag to indicate reporting status will solve the first
>>> issue and to an extent the second as well. Having refcnt will possibly solve
>>> both of them. What I am wondering about is that in my case I could easily
>>> impact the performance negatively by performing more bitmap scanning.
>> I realized that it may not be possible for me to directly adopt either refcnt
>> or zone flags just because of the way I have page reporting setup right now.
>> For now, I will just replace the refcnt with a bitflag as that should work
>> for most of the cases.  Nevertheless, I will also keep looking for a better way.
> If nothing else something you could consider is a refcnt for the
> number of bits you have set in your bitfield. Then all you would need
> to be doing is replace the cmpxchg with just a atomic_fetch_inc and
> what you would need to do is have your worker thread track how many
> bits it has cleared and subtract that from the refcnt at the end.

Thanks, I will think about this suggestion as well.

Based on your previous suggestion and what you have already proposed in your
series I can think of a way to atleast ensure reporting for zones freeing pages
after getting scanned in wq.
(In case I decide to go ahead with this approach I will mention that this change
is based on your series. Please do let me know if there is a better way to give

However, a situation where the same zone is reporting pages from the bitmap
section already scanned with zero freeing activity on other zones, may not
be entirely handled.

In any case, I think what I have in my mind will be better than what I have
right now. I will try to implement and test it to see if it can actually work.


  reply index

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-12 13:12 [RFC][PATCH v12 0/2] mm: Support for page reporting Nitesh Narayan Lal
2019-08-12 13:12 ` [RFC][Patch v12 1/2] mm: page_reporting: core infrastructure Nitesh Narayan Lal
2019-08-12 18:47   ` Alexander Duyck
2019-08-12 20:04     ` Nitesh Narayan Lal
2019-08-20 14:11       ` Nitesh Narayan Lal
2019-08-12 20:05     ` David Hildenbrand
2019-08-13 10:30       ` Nitesh Narayan Lal
2019-08-13 10:34         ` David Hildenbrand
2019-08-13 10:42           ` Nitesh Narayan Lal
2019-08-13 10:44             ` David Hildenbrand
2019-08-13 23:14           ` Alexander Duyck
2019-08-14  7:07             ` David Hildenbrand
2019-08-14 12:49               ` [virtio-dev] " Nitesh Narayan Lal
2019-08-14 15:49     ` Nitesh Narayan Lal
2019-08-14 16:11       ` Alexander Duyck
2019-08-15 13:15         ` Nitesh Narayan Lal
2019-08-15 19:22           ` Nitesh Narayan Lal
2019-08-15 23:00             ` Alexander Duyck
2019-08-16 18:35               ` Nitesh Narayan Lal [this message]
2019-08-30 15:15     ` Nitesh Narayan Lal
2019-08-30 15:31       ` Alexander Duyck
2019-08-30 16:05         ` Nitesh Narayan Lal
2019-09-04  8:40           ` [virtio-dev] " David Hildenbrand
2019-10-10 20:36   ` Alexander Duyck
2019-10-11 11:02     ` Nitesh Narayan Lal
2019-08-12 13:12 ` [RFC][Patch v12 2/2] virtio-balloon: interface to support free page reporting Nitesh Narayan Lal
2019-08-14 10:29   ` Cornelia Huck
2019-08-14 11:47     ` Nitesh Narayan Lal
2019-08-14 13:42       ` Cornelia Huck
2019-08-14 14:01         ` Nitesh Narayan Lal
2019-08-12 13:13 ` [QEMU Patch 1/2] virtio-balloon: adding bit for page reporting support Nitesh Narayan Lal
2019-08-12 13:13   ` [QEMU Patch 2/2] virtio-balloon: support for handling page reporting Nitesh Narayan Lal
2019-08-12 15:18     ` Alexander Duyck
2019-08-12 15:26       ` Nitesh Narayan Lal
2019-09-11 12:30 ` [RFC][PATCH v12 0/2] mm: Support for " David Hildenbrand

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cc681772-40dd-d429-1715-96498fa9c48e@redhat.com \
    --to=nitesh@redhat.com \
    --cc=aarcange@redhat.com \
    --cc=alexander.duyck@gmail.com \
    --cc=cohuck@redhat.com \
    --cc=dave.hansen@intel.com \
    --cc=david@redhat.com \
    --cc=dhildenb@redhat.com \
    --cc=dodgen@google.com \
    --cc=john.starks@microsoft.com \
    --cc=konrad.wilk@oracle.com \
    --cc=kvm@vger.kernel.org \
    --cc=lcapitulino@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@suse.com \
    --cc=mst@redhat.com \
    --cc=pagupta@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=riel@surriel.com \
    --cc=virtio-dev@lists.oasis-open.org \
    --cc=wei.w.wang@intel.com \
    --cc=yang.zhang.wz@gmail.com \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

KVM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kvm/0 kvm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kvm kvm/ https://lore.kernel.org/kvm \
	public-inbox-index kvm

Example config snippet for mirrors

Newsgroup available over NNTP:

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git