From: Rashmica Gupta <rashmica.g@gmail.com>
To: Oscar Salvador <osalvador@suse.de>
Cc: David Hildenbrand <david@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
Michal Hocko <mhocko@suse.com>,
Dan Williams <dan.j.williams@intel.com>,
pasha.tatashin@soleen.com, Jonathan.Cameron@huawei.com,
anshuman.khandual@arm.com, Vlastimil Babka <vbabka@suse.cz>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 0/5] Allocate memmap from hotadded memory
Date: Wed, 10 Jul 2019 11:14:25 +1000 [thread overview]
Message-ID: <CAC6rBsn5Q1kEB3z2f+wuCfY+=UPWUTRi5Eqyr8GNsv9+BmmDjw@mail.gmail.com> (raw)
In-Reply-To: <CAC6rBskRyh5Tj9L-6T4dTgA18H0Y8GsMdC-X5_0Jh1SVfLLYtg@mail.gmail.com>
Woops, looks like my phone doesn't send plain text emails :/
On Tue, Jul 2, 2019 at 6:52 PM Rashmica Gupta <rashmica.g@gmail.com> wrote:
>
> On Tue, Jul 2, 2019 at 5:48 PM Oscar Salvador <osalvador@suse.de> wrote:
>>
>> On Tue, Jul 02, 2019 at 04:42:34PM +1000, Rashmica Gupta wrote:
>> > Hi David,
>> >
>> > Sorry for the late reply.
>> >
>> > On Wed, 2019-06-26 at 10:28 +0200, David Hildenbrand wrote:
>> > > On 26.06.19 10:15, Oscar Salvador wrote:
>> > > > On Wed, Jun 26, 2019 at 10:11:06AM +0200, David Hildenbrand wrote:
>> > > > > Back then, I already mentioned that we might have some users that
>> > > > > remove_memory() they never added in a granularity it wasn't
>> > > > > added. My
>> > > > > concerns back then were never fully sorted out.
>> > > > >
>> > > > > arch/powerpc/platforms/powernv/memtrace.c
>> > > > >
>> > > > > - Will remove memory in memory block size chunks it never added
>> > > > > - What if that memory resides on a DIMM added via
>> > > > > MHP_MEMMAP_DEVICE?
>> > > > >
>> > > > > Will it at least bail out? Or simply break?
>> > > > >
>> > > > > IOW: I am not yet 100% convinced that MHP_MEMMAP_DEVICE is save
>> > > > > to be
>> > > > > introduced.
>> > > >
>> > > > Uhm, I will take a closer look and see if I can clear your
>> > > > concerns.
>> > > > TBH, I did not try to use arch/powerpc/platforms/powernv/memtrace.c
>> > > > yet.
>> > > >
>> > > > I will get back to you once I tried it out.
>> > > >
>> > >
>> > > BTW, I consider the code in arch/powerpc/platforms/powernv/memtrace.c
>> > > very ugly and dangerous.
>> >
>> > Yes it would be nice to clean this up.
>> >
>> > > We should never allow to manually
>> > > offline/online pages / hack into memory block states.
>> > >
>> > > What I would want to see here is rather:
>> > >
>> > > 1. User space offlines the blocks to be used
>> > > 2. memtrace installs a hotplug notifier and hinders the blocks it
>> > > wants
>> > > to use from getting onlined.
>> > > 3. memory is not added/removed/onlined/offlined in memtrace code.
>> > >
>> >
>> > I remember looking into doing it a similar way. I can't recall the
>> > details but my issue was probably 'how does userspace indicate to
>> > the kernel that this memory being offlined should be removed'?
>> >
>> > I don't know the mm code nor how the notifiers work very well so I
>> > can't quite see how the above would work. I'm assuming memtrace would
>> > register a hotplug notifier and when memory is offlined from userspace,
>> > the callback func in memtrace would be called if the priority was high
>> > enough? But how do we know that the memory being offlined is intended
>> > for usto touch? Is there a way to offline memory from userspace not
>> > using sysfs or have I missed something in the sysfs interface?
>> >
>> > On a second read, perhaps you are assuming that memtrace is used after
>> > adding new memory at runtime? If so, that is not the case. If not, then
>> > would you be able to clarify what I'm not seeing?
>>
>> Hi Rashmica,
>>
>> let us go the easy way here.
>> Could you please explain:
>>
>
> Sure!
>
>>
>> 1) How memtrace works
>
>
> You write the size of the chunk of memory you want into the debugfs file
> and memtrace will attempt to find a contiguous section of memory of that size
> that can be offlined. If it finds that, then the memory is removed from the
> kernel's mappings. If you want a different size, then you write that to the
> debugsfs file and memtrace will re-add the memory it first removed and then
> try to offline and remove the a chunk of the new size.
>
>
>>
>> 2) Why it was designed, what is the goal of the interface?
>> 3) When it is supposed to be used?
>>
>
> There is a hardware debugging facility (htm) on some power chips. To use
> this you need a contiguous portion of memory for the output to be dumped
> to - and we obviously don't want this memory to be simultaneously used by
> the kernel.
>
> At boot time we can portion off a section of memory for this (and not tell the
> kernel about it), but sometimes you want to be able to use the hardware
> debugging facilities and you haven't done this and you don't want to reboot
> your machine - and memtrace is the solution for this.
>
> If you're curious one tool that uses this debugging facility is here:
> https://github.com/open-power/pdbg. Relevant files are libpdbg/htm.c and src/htm.c.
>
>
>> I have seen a couple of reports in the past from people running memtrace
>> and failing to do so sometimes, and back then I could not grasp why people
>> was using it, or under which circumstances was nice to have.
>> So it would be nice to have a detailed explanation from the person who wrote
>> it.
>>
>
> Is that enough detail?
>
>>
>> Thanks
>>
>> --
>> Oscar Salvador
>> SUSE L3
next prev parent reply other threads:[~2019-07-10 1:14 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-25 7:52 [PATCH v2 0/5] Allocate memmap from hotadded memory Oscar Salvador
2019-06-25 7:52 ` [PATCH v2 1/5] drivers/base/memory: Remove unneeded check in remove_memory_block_devices Oscar Salvador
2019-06-25 8:01 ` David Hildenbrand
2019-06-25 8:03 ` David Hildenbrand
2019-06-25 8:09 ` Oscar Salvador
2019-06-25 8:27 ` David Hildenbrand
2019-06-25 7:52 ` [PATCH v2 2/5] mm,memory_hotplug: Introduce MHP_VMEMMAP_FLAGS Oscar Salvador
2019-06-25 8:31 ` David Hildenbrand
2019-07-24 20:11 ` Dan Williams
2019-07-24 21:36 ` osalvador
2019-07-25 9:27 ` Oscar Salvador
2019-07-25 9:30 ` David Hildenbrand
2019-07-25 9:40 ` Oscar Salvador
2019-07-25 10:04 ` David Hildenbrand
2019-07-25 10:13 ` Oscar Salvador
2019-07-25 10:15 ` David Hildenbrand
2019-06-25 7:52 ` [PATCH v2 3/5] mm,memory_hotplug: Introduce Vmemmap page helpers Oscar Salvador
2019-06-25 10:28 ` David Hildenbrand
2019-06-26 9:48 ` Oscar Salvador
2019-06-25 7:52 ` [PATCH v2 4/5] mm,memory_hotplug: allocate memmap from the added memory range for sparse-vmemmap Oscar Salvador
2019-06-25 8:49 ` David Hildenbrand
2019-06-26 8:13 ` Oscar Salvador
2019-06-26 8:15 ` David Hildenbrand
2019-06-26 8:17 ` Anshuman Khandual
2019-06-26 8:28 ` Oscar Salvador
2019-07-24 21:49 ` Dan Williams
2019-06-25 7:52 ` [PATCH v2 5/5] mm,memory_hotplug: Allow userspace to enable/disable vmemmap Oscar Salvador
2019-06-25 8:25 ` [PATCH v2 0/5] Allocate memmap from hotadded memory David Hildenbrand
2019-06-25 8:33 ` David Hildenbrand
2019-06-26 8:03 ` Oscar Salvador
2019-06-26 8:11 ` David Hildenbrand
2019-06-26 8:15 ` Oscar Salvador
2019-06-26 8:27 ` Oscar Salvador
2019-06-26 8:37 ` David Hildenbrand
2019-06-26 8:28 ` David Hildenbrand
2019-07-02 6:42 ` Rashmica Gupta
2019-07-02 7:48 ` Oscar Salvador
2019-07-02 8:52 ` Rashmica Gupta
2019-07-10 1:14 ` Rashmica Gupta [this message]
2019-07-31 12:08 ` Michal Hocko
2019-07-31 23:06 ` Rashmica Gupta
2019-08-01 7:17 ` Michal Hocko
2019-08-01 7:18 ` David Hildenbrand
2019-08-01 7:24 ` Michal Hocko
2019-08-01 7:26 ` David Hildenbrand
2019-08-01 7:31 ` David Hildenbrand
2019-08-01 7:39 ` Michal Hocko
2019-08-01 7:48 ` Michal Hocko
2019-08-01 9:18 ` David Hildenbrand
2019-08-01 7:34 ` Michal Hocko
2019-08-01 7:50 ` David Hildenbrand
2019-08-01 8:04 ` Michal Hocko
2019-07-16 12:28 ` David Hildenbrand
2019-07-29 5:42 ` Rashmica Gupta
2019-07-29 8:06 ` David Hildenbrand
2019-07-30 7:08 ` Rashmica Gupta
2019-07-31 2:21 ` Rashmica Gupta
2019-07-31 9:39 ` David Hildenbrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAC6rBsn5Q1kEB3z2f+wuCfY+=UPWUTRi5Eqyr8GNsv9+BmmDjw@mail.gmail.com' \
--to=rashmica.g@gmail.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=akpm@linux-foundation.org \
--cc=anshuman.khandual@arm.com \
--cc=dan.j.williams@intel.com \
--cc=david@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=osalvador@suse.de \
--cc=pasha.tatashin@soleen.com \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).