From: Mike Kravetz <mike.kravetz@oracle.com>
To: David Hildenbrand <david@redhat.com>,
Mina Almasry <almasrymina@google.com>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
Michal Privoznik <mprivozn@redhat.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
Michal Hocko <mhocko@kernel.org>,
Muchun Song <songmuchun@bytedance.com>,
"Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>,
Tejun Heo <tj@kernel.org>
Subject: Re: cgroup and FALLOC_FL_PUNCH_HOLE: WARNING: CPU: 13 PID: 2438 at mm/page_counter.c:57 page_counter_uncharge+0x4b/0x5
Date: Wed, 14 Oct 2020 11:31:16 -0700 [thread overview]
Message-ID: <5f196069-8b98-0ad3-55e8-19af03d715cd@oracle.com> (raw)
In-Reply-To: <65a1946f-dbf9-5767-5b51-9c1b786051d1@redhat.com>
On 10/14/20 11:18 AM, David Hildenbrand wrote:
> On 14.10.20 19:56, Mina Almasry wrote:
>> On Wed, Oct 14, 2020 at 9:15 AM David Hildenbrand <david@redhat.com> wrote:
>>>
>>> On 14.10.20 17:22, David Hildenbrand wrote:
>>>> Hi everybody,
>>>>
>>>> Michal Privoznik played with "free page reporting" in QEMU/virtio-balloon
>>>> with hugetlbfs and reported that this results in [1]
>>>>
>>>> 1. WARNING: CPU: 13 PID: 2438 at mm/page_counter.c:57 page_counter_uncharge+0x4b/0x5
>>>>
>>>> 2. Any hugetlbfs allocations failing. (I assume because some accounting is wrong)
>>>>
>>>>
>>>> QEMU with free page hinting uses fallocate(FALLOC_FL_PUNCH_HOLE)
>>>> to discard pages that are reported as free by a VM. The reporting
>>>> granularity is in pageblock granularity. So when the guest reports
>>>> 2M chunks, we fallocate(FALLOC_FL_PUNCH_HOLE) one huge page in QEMU.
>>>>
>>>> I was also able to reproduce (also with virtio-mem, which similarly
>>>> uses fallocate(FALLOC_FL_PUNCH_HOLE)) on latest v5.9
>>>> (and on v5.7.X from F32).
>>>>
>>>> Looks like something with fallocate(FALLOC_FL_PUNCH_HOLE) accounting
>>>> is broken with cgroups. I did *not* try without cgroups yet.
>>>>
>>>> Any ideas?
>>
>> Hi David,
>>
>> I may be able to dig in and take a look. How do I reproduce this
>> though? I just fallocate(FALLOC_FL_PUNCH_HOLE) one 2MB page in a
>> hugetlb region?
>>
>
> Hi Mina,
>
> thanks for having a look. I started poking around myself but,
> being new to cgroup code, I even failed to understand why that code gets
> triggered though the hugetlb controller isn't even enabled.
>
> I assume you at least have to make sure that there is
> a page populated (MMAP_POPULATE, or read/write it). But I am not
> sure yet if a single fallocate(FALLOC_FL_PUNCH_HOLE) is
> sufficient, or if it will require a sequence of
> populate+discard(punch) (or multi-threading).
FWIW - I ran libhugetlbfs tests which do a bunch of hole punching
with (and without) hugetlb controller enabled and did not see this issue.
May need to reproduce via QEMU as below.
--
Mike Kravetz
> What definitely makes it trigger is via QEMU
>
> qemu-system-x86_64 \
> -machine pc-i440fx-4.0,accel=kvm,usb=off,dump-guest-core=off,memory-backend=pc.ram \
> -cpu host,migratable=on \
> -m 4096 \
> -object memory-backend-memfd,id=pc.ram,hugetlb=yes,hugetlbsize=2097152,size=4294967296 \
> -overcommit mem-lock=off \
> -smp 4,sockets=1,dies=1,cores=2,threads=2 \
> -nodefaults \
> -nographic \
> -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x4 \
> -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 \
> -blockdev '{"driver":"file","filename":"../Fedora-Cloud-Base-32-1.6.x86_64.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
> -blockdev '{"node-name":"libvirt-1-format","read-only":false,"discard":"unmap","driver":"qcow2","file":"libvirt-1-storage","backing":null}' \
> -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=libvirt-1-format,id=scsi0-0-0-0,bootindex=1 \
> -chardev stdio,nosignal,id=serial \
> -device isa-serial,chardev=serial \
> -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7,free-page-reporting=on
>
>
> However, you need a recent QEMU (>= v5.1 IIRC) and a recent kernel
> (>= v5.7) inside your guest image.
>
> Fedora rawhide qcow2 should do: https://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/Cloud/x86_64/images/Fedora-Cloud-Base-Rawhide-20201004.n.1.x86_64.qcow2
>
next prev parent reply other threads:[~2020-10-14 18:33 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-14 15:22 cgroup and FALLOC_FL_PUNCH_HOLE: WARNING: CPU: 13 PID: 2438 at mm/page_counter.c:57 page_counter_uncharge+0x4b/0x5 David Hildenbrand
2020-10-14 16:15 ` David Hildenbrand
2020-10-14 17:56 ` Mina Almasry
2020-10-14 18:18 ` David Hildenbrand
2020-10-14 18:31 ` Mike Kravetz [this message]
2020-10-15 7:56 ` David Hildenbrand
2020-10-15 8:57 ` David Hildenbrand
2020-10-15 9:01 ` David Hildenbrand
2020-10-15 23:14 ` Mike Kravetz
2020-10-20 13:38 ` David Hildenbrand
2020-10-21 3:35 ` Mike Kravetz
2020-10-21 12:42 ` David Hildenbrand
2020-10-21 12:57 ` Michal Privoznik
2020-10-21 13:11 ` David Hildenbrand
2020-10-21 13:34 ` David Hildenbrand
2020-10-21 13:38 ` David Hildenbrand
2020-10-21 16:58 ` Mike Kravetz
2020-10-21 17:30 ` David Hildenbrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5f196069-8b98-0ad3-55e8-19af03d715cd@oracle.com \
--to=mike.kravetz@oracle.com \
--cc=almasrymina@google.com \
--cc=aneesh.kumar@linux.vnet.ibm.com \
--cc=david@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=mprivozn@redhat.com \
--cc=mst@redhat.com \
--cc=songmuchun@bytedance.com \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).