linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: qixiaoyu1 <qxy65535@gmail.com>
To: Jaegeuk Kim <jaegeuk@kernel.org>, Chao Yu <chao@kernel.org>
Cc: linux-kernel@vger.kernel.org,
	linux-f2fs-devel@lists.sourceforge.net,
	qixiaoyu1 <qixiaoyu1@xiaomi.com>,
	xiongping1 <xiongping1@xiaomi.com>
Subject: [PATCH 0/5] Support enhanced hot/cold data separation for f2fs
Date: Mon, 28 Nov 2022 16:58:54 +0800	[thread overview]
Message-ID: <20221128085859.5295-1-qixiaoyu1@xiaomi.com> (raw)

This patch series introduce a runtime hot/cold data separation
method for f2fs, in order to improve the accuracy for data
temperature classification, reduce the garbage collection overhead
after long-term data updates.

Enhanced hot/cold data separation can record data block update
frequency as "age" of the extent per inode, and take use of the age
info to indicate better temperature type for data block allocation:
 - It record total data blocks allocated since mount;
 - When file extent has been updated, it calculate the count of data
blocks allocated since last update as the age of the extent;
 - Before the data block allocated, it search for the age info and
choose the suitable segment for allocation.

Patch 1 records total data blocks allocated since mount.

Patch 2 implements cache to manager block update frequency per inode.

Patch 3 adds age_extent_cache mount option to enable this feature only
when age_extent_cache mount option is on.

Patch 4 updates block age info during out of place update.

Patch 5 implements data block seperation with block update frequency.

Test and result:
 - Prepare: create about 30000 files
  * 3% for cold files (with cold file extension like .apk, from 3M to 10M)
  * 50% for warm files (with random file extension like .FcDxq, from 1K
to 4M)
  * 47% for hot files (with hot file extension like .db, from 1K to 256K)
 - create(5%)/random update(90%)/delete(5%) the files
  * total write amount is about 70G
  * fsync will be called for .db files, and buffered write will be used
for other files

The storage of test device is large enough(128G) so that it will not
switch to SSR mode during the test.

Benefit: dirty segment count increment reduce about 14%
 - before: Dirty +21110
 - after:  Dirty +18286

qixiaoyu1 (2):
  f2fs: update block age info during out of place update
  f2fs: implement data block seperation with block update frequency

xiongping1 (3):
  f2fs: record total data blocks allocated since mount
  f2fs: implement cache to manager block update frequency per inode
  f2fs: add age_extent_cache mount option

 Documentation/ABI/testing/sysfs-fs-f2fs |  14 +
 Documentation/filesystems/f2fs.rst      |   4 +
 fs/f2fs/Kconfig                         |   7 +
 fs/f2fs/Makefile                        |   1 +
 fs/f2fs/block_age.c                     | 733 ++++++++++++++++++++++++
 fs/f2fs/debug.c                         |  20 +
 fs/f2fs/f2fs.h                          |  83 +++
 fs/f2fs/file.c                          |  10 +
 fs/f2fs/inode.c                         |   8 +
 fs/f2fs/namei.c                         |   4 +
 fs/f2fs/node.c                          |   7 +-
 fs/f2fs/node.h                          |   3 +
 fs/f2fs/segment.c                       |  23 +
 fs/f2fs/shrinker.c                      |   3 +
 fs/f2fs/super.c                         |  51 ++
 fs/f2fs/sysfs.c                         |  28 +
 include/trace/events/f2fs.h             | 239 ++++++++
 17 files changed, 1237 insertions(+), 1 deletion(-)
 create mode 100644 fs/f2fs/block_age.c

-- 
2.36.1


             reply	other threads:[~2022-11-28  9:06 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-11-28  8:58 qixiaoyu1 [this message]
2022-11-28  8:58 ` [PATCH 1/5] f2fs: record total data blocks allocated since mount qixiaoyu1
2022-11-29  2:51   ` Jaegeuk Kim
2022-12-05 19:04     ` [f2fs-dev] " Jaegeuk Kim
2022-11-28  8:58 ` [PATCH 2/5] f2fs: implement cache to manager block update frequency per inode qixiaoyu1
2022-11-28 10:44   ` kernel test robot
2022-11-28  8:58 ` [PATCH 3/5] f2fs: add age_extent_cache mount option qixiaoyu1
2022-11-28  8:58 ` [PATCH 4/5] f2fs: update block age info during out of place update qixiaoyu1
2022-11-28 11:14   ` kernel test robot
2022-12-12  3:04   ` kernel test robot
2022-11-28  8:58 ` [PATCH 5/5] f2fs: implement data block seperation with block update frequency qixiaoyu1
2022-11-29  7:49 ` [PATCH] f2fs: Support enhanced hot/cold data separation for f2fs Yangtao Li

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20221128085859.5295-1-qixiaoyu1@xiaomi.com \
    --to=qxy65535@gmail.com \
    --cc=chao@kernel.org \
    --cc=jaegeuk@kernel.org \
    --cc=linux-f2fs-devel@lists.sourceforge.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=qixiaoyu1@xiaomi.com \
    --cc=xiongping1@xiaomi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).