[f2fs-dev] [PATCH] f2fs: set *_data_age_threshold according to user_block_count

[f2fs-dev] [PATCH] f2fs: set *_data_age_threshold according to user_block_count

[Yangtao Li via Linux-f2fs-devel]

Commit 71644dff4811 ("f2fs: add block_age-based extent cache")
introduce age extent cache, which experimental data is based on
a 128G storage device, and hot and warm data age threshold are
set to 1G and 10G respectively. But it is unreasonable to set
this value to 1G or 10G by default, which varies depending on
the environment. For small storage devices, some storage devices
do not even have 10G.

Let's change hot and warm data age threshold to 1% and 10% of
user_block_count respectively.

Signed-off-by: Yangtao Li <frank.li@vivo.com>
---
 Documentation/ABI/testing/sysfs-fs-f2fs | 6 ++----
 fs/f2fs/extent_cache.c                  | 2 --
 fs/f2fs/f2fs.h                          | 9 +++++----
 fs/f2fs/super.c                         | 2 ++
 4 files changed, 9 insertions(+), 10 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-fs-f2fs b/Documentation/ABI/testing/sysfs-fs-f2fs
index 75420c242cc4..c7952f1baf59 100644
--- a/Documentation/ABI/testing/sysfs-fs-f2fs
+++ b/Documentation/ABI/testing/sysfs-fs-f2fs
@@ -660,15 +660,13 @@ What:		/sys/fs/f2fs/<disk>/hot_data_age_threshold
 Date:		November 2022
 Contact:	"Ping Xiong" <xiongping1@xiaomi.com>
 Description:	When DATA SEPARATION is on, it controls the age threshold to indicate
-		the data blocks as hot. By default it was initialized as 262144 blocks
-		(equals to 1GB).
+		the data blocks as hot. By default it was initialized as 1% of user_block_count.
 
 What:		/sys/fs/f2fs/<disk>/warm_data_age_threshold
 Date:		November 2022
 Contact:	"Ping Xiong" <xiongping1@xiaomi.com>
 Description:	When DATA SEPARATION is on, it controls the age threshold to indicate
-		the data blocks as warm. By default it was initialized as 2621440 blocks
-		(equals to 10GB).
+		the data blocks as warm. By default it was initialized as 10% of user_block_count.
 
 What:		/sys/fs/f2fs/<disk>/fault_rate
 Date:		May 2016
diff --git a/fs/f2fs/extent_cache.c b/fs/f2fs/extent_cache.c
index 1daf8c88c09b..9c7e304d5660 100644
--- a/fs/f2fs/extent_cache.c
+++ b/fs/f2fs/extent_cache.c
@@ -1235,8 +1235,6 @@ void f2fs_init_extent_cache_info(struct f2fs_sb_info *sbi)
 
 	/* initialize for block age extents */
 	atomic64_set(&sbi->allocated_data_blocks, 0);
-	sbi->hot_data_age_threshold = DEF_HOT_DATA_AGE_THRESHOLD;
-	sbi->warm_data_age_threshold = DEF_WARM_DATA_AGE_THRESHOLD;
 }
 
 int __init f2fs_create_extent_cache(void)
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index f3c5f7740c1a..3b853c302a43 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -615,11 +615,12 @@ enum {
 #define SAME_AGE_REGION			1024
 
 /*
- * Define data block with age less than 1GB as hot data
- * define data block with age less than 10GB but more than 1GB as warm data
+ * Define data block with age less than 1% of user_block_count as hot data
+ * Define data block with age less than 10% of user_block_count but more
+ * than 1% of user_block_count as warm data
  */
-#define DEF_HOT_DATA_AGE_THRESHOLD	262144
-#define DEF_WARM_DATA_AGE_THRESHOLD	2621440
+#define DEF_HOT_DATA_AGE_THRESHOLD	1
+#define DEF_WARM_DATA_AGE_THRESHOLD	10
 
 /* extent cache type */
 enum extent_type {
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 5fc83771042d..8333ea5b8ffd 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -4088,6 +4088,8 @@ static void f2fs_tuning_parameters(struct f2fs_sb_info *sbi)
 					BIT(F2FS_IPU_HONOR_OPU_WRITE);
 	}
 
+	sbi->hot_data_age_threshold = sbi->user_block_count * DEF_HOT_DATA_AGE_THRESHOLD / 100;
+	sbi->warm_data_age_threshold = sbi->user_block_count * DEF_WARM_DATA_AGE_THRESHOLD / 100;
 	sbi->readdir_ra = true;
 }
 
-- 
2.25.1



_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH] f2fs: set *_data_age_threshold according to user_block_count

[qixiaoyu]

On Tue, Jan 17, 2023 at 06:30:42PM +0800, Yangtao Li via Linux-f2fs-devel wrote:
> Commit 71644dff4811 ("f2fs: add block_age-based extent cache")
> introduce age extent cache, which experimental data is based on
> a 128G storage device, and hot and warm data age threshold are
> set to 1G and 10G respectively. But it is unreasonable to set
> this value to 1G or 10G by default, which varies depending on
> the environment. For small storage devices, some storage devices
> do not even have 10G.
> 
> Let's change hot and warm data age threshold to 1% and 10% of
> user_block_count respectively.
> 

Hi Yangtao,

Thanks for your patch.

The block age here refers to total data blocks allocated of filesystem
between two consecutive updates. So, it has nothing to do with storage
size.

> Signed-off-by: Yangtao Li <frank.li@vivo.com>
> ---
>  Documentation/ABI/testing/sysfs-fs-f2fs | 6 ++----
>  fs/f2fs/extent_cache.c                  | 2 --
>  fs/f2fs/f2fs.h                          | 9 +++++----
>  fs/f2fs/super.c                         | 2 ++
>  4 files changed, 9 insertions(+), 10 deletions(-)
> 
> diff --git a/Documentation/ABI/testing/sysfs-fs-f2fs b/Documentation/ABI/testing/sysfs-fs-f2fs
> index 75420c242cc4..c7952f1baf59 100644
> --- a/Documentation/ABI/testing/sysfs-fs-f2fs
> +++ b/Documentation/ABI/testing/sysfs-fs-f2fs
> @@ -660,15 +660,13 @@ What:		/sys/fs/f2fs/<disk>/hot_data_age_threshold
>  Date:		November 2022
>  Contact:	"Ping Xiong" <xiongping1@xiaomi.com>
>  Description:	When DATA SEPARATION is on, it controls the age threshold to indicate
> -		the data blocks as hot. By default it was initialized as 262144 blocks
> -		(equals to 1GB).
> +		the data blocks as hot. By default it was initialized as 1% of user_block_count.
>  
>  What:		/sys/fs/f2fs/<disk>/warm_data_age_threshold
>  Date:		November 2022
>  Contact:	"Ping Xiong" <xiongping1@xiaomi.com>
>  Description:	When DATA SEPARATION is on, it controls the age threshold to indicate
> -		the data blocks as warm. By default it was initialized as 2621440 blocks
> -		(equals to 10GB).
> +		the data blocks as warm. By default it was initialized as 10% of user_block_count.
>  
>  What:		/sys/fs/f2fs/<disk>/fault_rate
>  Date:		May 2016
> diff --git a/fs/f2fs/extent_cache.c b/fs/f2fs/extent_cache.c
> index 1daf8c88c09b..9c7e304d5660 100644
> --- a/fs/f2fs/extent_cache.c
> +++ b/fs/f2fs/extent_cache.c
> @@ -1235,8 +1235,6 @@ void f2fs_init_extent_cache_info(struct f2fs_sb_info *sbi)
>  
>  	/* initialize for block age extents */
>  	atomic64_set(&sbi->allocated_data_blocks, 0);
> -	sbi->hot_data_age_threshold = DEF_HOT_DATA_AGE_THRESHOLD;
> -	sbi->warm_data_age_threshold = DEF_WARM_DATA_AGE_THRESHOLD;
>  }
>  
>  int __init f2fs_create_extent_cache(void)
> diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
> index f3c5f7740c1a..3b853c302a43 100644
> --- a/fs/f2fs/f2fs.h
> +++ b/fs/f2fs/f2fs.h
> @@ -615,11 +615,12 @@ enum {
>  #define SAME_AGE_REGION			1024
>  
>  /*
> - * Define data block with age less than 1GB as hot data
> - * define data block with age less than 10GB but more than 1GB as warm data
> + * Define data block with age less than 1% of user_block_count as hot data
> + * Define data block with age less than 10% of user_block_count but more
> + * than 1% of user_block_count as warm data
>   */
> -#define DEF_HOT_DATA_AGE_THRESHOLD	262144
> -#define DEF_WARM_DATA_AGE_THRESHOLD	2621440
> +#define DEF_HOT_DATA_AGE_THRESHOLD	1
> +#define DEF_WARM_DATA_AGE_THRESHOLD	10
>  
>  /* extent cache type */
>  enum extent_type {
> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
> index 5fc83771042d..8333ea5b8ffd 100644
> --- a/fs/f2fs/super.c
> +++ b/fs/f2fs/super.c
> @@ -4088,6 +4088,8 @@ static void f2fs_tuning_parameters(struct f2fs_sb_info *sbi)
>  					BIT(F2FS_IPU_HONOR_OPU_WRITE);
>  	}
>  
> +	sbi->hot_data_age_threshold = sbi->user_block_count * DEF_HOT_DATA_AGE_THRESHOLD / 100;
> +	sbi->warm_data_age_threshold = sbi->user_block_count * DEF_WARM_DATA_AGE_THRESHOLD / 100;
>  	sbi->readdir_ra = true;
>  }
>  
> -- 
> 2.25.1
> 
> 
> 
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] f2fs: set *_data_age_threshold according to user_block_count

[Yangtao Li via Linux-f2fs-devel]

Hi qixiaoyu,

> The block age here refers to total data blocks allocated of filesystem between two consecutive updates.

Yes, you are right.

> So, it has nothing to do with storage size.

But I think that the total data blocks allocated of filesystem between two consecutive updates
has something to do with the storage size. For example, for a 60M f2fs image, the lifetime_write_kbytes
will hardly reach 10G, or even 1G.

Thx,
Yangtao


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] f2fs: set *_data_age_threshold according to user_block_count

[qixiaoyu]

On Tue, Jan 17, 2023 at 09:38:14PM +0800, Yangtao Li wrote:
> Hi qixiaoyu,
> 
> > The block age here refers to total data blocks allocated of filesystem between two consecutive updates.
> 
> Yes, you are right.
> 
> > So, it has nothing to do with storage size.
> 
> But I think that the total data blocks allocated of filesystem between two consecutive updates
> has something to do with the storage size. For example, for a 60M f2fs image, the lifetime_write_kbytes
> will hardly reach 10G, or even 1G.
> 
> Thx,
> Yangtao

Hi Yangtao,

Block update frequency may related to applications and usage patterns,
not storage size. A 1G f2fs image may have a similar block age to a
10G f2fs image when running the same program.

So, it might not be a good idea to decide the *_data_age_threshold
based on user_block_count.

Thanks,


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel