From: Andrew Morton <akpm@linux-foundation.org>
To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org,
linux-mm@kvack.org, lkp@intel.com, longman@redhat.com,
mhocko@kernel.org, mm-commits@vger.kernel.org,
shakeelb@google.com, songmuchun@bytedance.com,
torvalds@linux-foundation.org, vdavydov.dev@gmail.com
Subject: [patch 19/21] mm/memcg: relocate mod_objcg_mlstate(), get_obj_stock() and put_obj_stock()
Date: Fri, 10 Dec 2021 14:47:05 -0800 [thread overview]
Message-ID: <20211210224705.8Ze1Q8nWt%akpm@linux-foundation.org> (raw)
In-Reply-To: <20211210144539.663efee2c80d8450e6180230@linux-foundation.org>
From: Waiman Long <longman@redhat.com>
Subject: mm/memcg: relocate mod_objcg_mlstate(), get_obj_stock() and put_obj_stock()
All the calls to mod_objcg_mlstate(), get_obj_stock() and put_obj_stock()
are done by functions defined within the same "#ifdef CONFIG_MEMCG_KMEM"
compilation block. When CONFIG_MEMCG_KMEM isn't defined, the following
compilation warnings will be issued [1] and [2].
mm/memcontrol.c:785:20: warning: unused function 'mod_objcg_mlstate'
mm/memcontrol.c:2113:33: warning: unused function 'get_obj_stock'
Fix these warning by moving those functions to under the same
CONFIG_MEMCG_KMEM compilation block. There is no functional change.
[1] https://lore.kernel.org/lkml/202111272014.WOYNLUV6-lkp@intel.com/
[2] https://lore.kernel.org/lkml/202111280551.LXsWYt1T-lkp@intel.com/
Link: https://lkml.kernel.org/r/20211129161140.306488-1-longman@redhat.com
Fixes: 559271146efc ("mm/memcg: optimize user context object stock access")
Fixes: 68ac5b3c8db2 ("mm/memcg: cache vmstat data in percpu memcg_stock_pcp")
Signed-off-by: Waiman Long <longman@redhat.com>
Reported-by: kernel test robot <lkp@intel.com>
Reviewed-by: Shakeel Butt <shakeelb@google.com>
Acked-by: Roman Gushchin <guro@fb.com>
Reviewed-by: Muchun Song <songmuchun@bytedance.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
mm/memcontrol.c | 106 +++++++++++++++++++++++-----------------------
1 file changed, 53 insertions(+), 53 deletions(-)
--- a/mm/memcontrol.c~mm-memcg-relocate-mod_objcg_mlstate-get_obj_stock-and-put_obj_stock
+++ a/mm/memcontrol.c
@@ -776,24 +776,6 @@ void __mod_lruvec_kmem_state(void *p, en
rcu_read_unlock();
}
-/*
- * mod_objcg_mlstate() may be called with irq enabled, so
- * mod_memcg_lruvec_state() should be used.
- */
-static inline void mod_objcg_mlstate(struct obj_cgroup *objcg,
- struct pglist_data *pgdat,
- enum node_stat_item idx, int nr)
-{
- struct mem_cgroup *memcg;
- struct lruvec *lruvec;
-
- rcu_read_lock();
- memcg = obj_cgroup_memcg(objcg);
- lruvec = mem_cgroup_lruvec(memcg, pgdat);
- mod_memcg_lruvec_state(lruvec, idx, nr);
- rcu_read_unlock();
-}
-
/**
* __count_memcg_events - account VM events in a cgroup
* @memcg: the memory cgroup
@@ -2137,41 +2119,6 @@ static bool obj_stock_flush_required(str
}
#endif
-/*
- * Most kmem_cache_alloc() calls are from user context. The irq disable/enable
- * sequence used in this case to access content from object stock is slow.
- * To optimize for user context access, there are now two object stocks for
- * task context and interrupt context access respectively.
- *
- * The task context object stock can be accessed by disabling preemption only
- * which is cheap in non-preempt kernel. The interrupt context object stock
- * can only be accessed after disabling interrupt. User context code can
- * access interrupt object stock, but not vice versa.
- */
-static inline struct obj_stock *get_obj_stock(unsigned long *pflags)
-{
- struct memcg_stock_pcp *stock;
-
- if (likely(in_task())) {
- *pflags = 0UL;
- preempt_disable();
- stock = this_cpu_ptr(&memcg_stock);
- return &stock->task_obj;
- }
-
- local_irq_save(*pflags);
- stock = this_cpu_ptr(&memcg_stock);
- return &stock->irq_obj;
-}
-
-static inline void put_obj_stock(unsigned long flags)
-{
- if (likely(in_task()))
- preempt_enable();
- else
- local_irq_restore(flags);
-}
-
/**
* consume_stock: Try to consume stocked charge on this cpu.
* @memcg: memcg to consume from.
@@ -2816,6 +2763,59 @@ retry:
*/
#define OBJCGS_CLEAR_MASK (__GFP_DMA | __GFP_RECLAIMABLE | __GFP_ACCOUNT)
+/*
+ * Most kmem_cache_alloc() calls are from user context. The irq disable/enable
+ * sequence used in this case to access content from object stock is slow.
+ * To optimize for user context access, there are now two object stocks for
+ * task context and interrupt context access respectively.
+ *
+ * The task context object stock can be accessed by disabling preemption only
+ * which is cheap in non-preempt kernel. The interrupt context object stock
+ * can only be accessed after disabling interrupt. User context code can
+ * access interrupt object stock, but not vice versa.
+ */
+static inline struct obj_stock *get_obj_stock(unsigned long *pflags)
+{
+ struct memcg_stock_pcp *stock;
+
+ if (likely(in_task())) {
+ *pflags = 0UL;
+ preempt_disable();
+ stock = this_cpu_ptr(&memcg_stock);
+ return &stock->task_obj;
+ }
+
+ local_irq_save(*pflags);
+ stock = this_cpu_ptr(&memcg_stock);
+ return &stock->irq_obj;
+}
+
+static inline void put_obj_stock(unsigned long flags)
+{
+ if (likely(in_task()))
+ preempt_enable();
+ else
+ local_irq_restore(flags);
+}
+
+/*
+ * mod_objcg_mlstate() may be called with irq enabled, so
+ * mod_memcg_lruvec_state() should be used.
+ */
+static inline void mod_objcg_mlstate(struct obj_cgroup *objcg,
+ struct pglist_data *pgdat,
+ enum node_stat_item idx, int nr)
+{
+ struct mem_cgroup *memcg;
+ struct lruvec *lruvec;
+
+ rcu_read_lock();
+ memcg = obj_cgroup_memcg(objcg);
+ lruvec = mem_cgroup_lruvec(memcg, pgdat);
+ mod_memcg_lruvec_state(lruvec, idx, nr);
+ rcu_read_unlock();
+}
+
int memcg_alloc_page_obj_cgroups(struct page *page, struct kmem_cache *s,
gfp_t gfp, bool new_page)
{
_
next prev parent reply other threads:[~2021-12-10 22:47 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-10 22:45 incoming Andrew Morton
2021-12-10 22:46 ` [patch 01/21] Increase default MLOCK_LIMIT to 8 MiB Andrew Morton
2021-12-11 1:08 ` Linus Torvalds
2021-12-10 22:46 ` [patch 02/21] MAINTAINERS: update kdump maintainers Andrew Morton
2021-12-10 22:46 ` [patch 03/21] mailmap: update email address for Guo Ren Andrew Morton
2021-12-10 22:46 ` [patch 04/21] filemap: remove PageHWPoison check from next_uptodate_page() Andrew Morton
2021-12-10 22:46 ` [patch 05/21] timers: implement usleep_idle_range() Andrew Morton
2021-12-10 22:46 ` [patch 06/21] mm/damon/core: fix fake load reports due to uninterruptible sleeps Andrew Morton
2021-12-10 22:46 ` [patch 07/21] mm/damon/core: use better timer mechanisms selection threshold Andrew Morton
2021-12-10 22:46 ` [patch 08/21] mm/damon/dbgfs: remove an unnecessary error message Andrew Morton
2021-12-10 22:46 ` [patch 09/21] mm/damon/core: remove unnecessary error messages Andrew Morton
2021-12-10 22:46 ` [patch 10/21] mm/damon/vaddr: remove an unnecessary warning message Andrew Morton
2021-12-10 22:46 ` [patch 11/21] mm/damon/vaddr-test: split a test function having >1024 bytes frame size Andrew Morton
2021-12-10 22:46 ` [patch 12/21] mm/damon/vaddr-test: remove unnecessary variables Andrew Morton
2021-12-10 22:46 ` [patch 13/21] selftests/damon: skip test if DAMON is running Andrew Morton
2021-12-10 22:46 ` [patch 14/21] selftests/damon: test DAMON enabling with empty target_ids case Andrew Morton
2021-12-10 22:46 ` [patch 15/21] selftests/damon: test wrong DAMOS condition ranges input Andrew Morton
2021-12-10 22:46 ` [patch 16/21] selftests/damon: test debugfs file reads/writes with huge count Andrew Morton
2021-12-10 22:46 ` [patch 17/21] selftests/damon: split test cases Andrew Morton
2021-12-10 22:47 ` [patch 18/21] mm/slub: fix endianness bug for alloc/free_traces attributes Andrew Morton
2021-12-10 22:47 ` Andrew Morton [this message]
2021-12-10 22:47 ` [patch 20/21] hugetlbfs: fix issue of preallocation of gigantic pages can't work Andrew Morton
2021-12-10 22:47 ` [patch 21/21] mm: bdi: initialize bdi_min_ratio when bdi is unregistered Andrew Morton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211210224705.8Ze1Q8nWt%akpm@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=guro@fb.com \
--cc=hannes@cmpxchg.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lkp@intel.com \
--cc=longman@redhat.com \
--cc=mhocko@kernel.org \
--cc=mm-commits@vger.kernel.org \
--cc=shakeelb@google.com \
--cc=songmuchun@bytedance.com \
--cc=torvalds@linux-foundation.org \
--cc=vdavydov.dev@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).