[RFC PATCH 0/2] apply write hints to select the type of segments

[RFC PATCH 0/2] apply write hints to select the type of segments

[Hyunchul Lee]

From: Hyunchul Lee <cheol.lee@lge.com>

Using write hints[1], applications can inform the life time of the data
written to devices. and this[2] reported that the write hints patch
decreased writes in NAND by 25%.

This hints help F2FS to determine the followings.
  1) the segment types where the data will be written.
  2) the hints that will be passed down to devices with the data of segments.

This patch set implements the first mapping from write hints to segment types
as shown below.

  hints                     segment type
  -----                     ------------
  WRITE_LIFE_SHORT          CURSEG_COLD_DATA
  WRITE_LIFE_EXTREME        CURSEG_HOT_DATA
  others                    CURSEG_WARM_DATA

The F2FS poliy for hot/cold seperation has precedence over this hints, And
hints are not applied in in-place update.

Before the second mapping is implemented, write hints are not passed down
to devices. Because it is better that the data of a segment have the same 
hint.

[1]: c75b1d9421f80f4143e389d2d50ddfc8a28c8c35
[2]: https://lwn.net/Articles/726477/

Hyunchul Lee (2):
  f2fs: apply write hints to select the type of segments for buffered
    write
  f2fs: apply write hints to select the type of segment for direct write

 fs/f2fs/data.c    | 101 ++++++++++++++++++++++++++++++++----------------------
 fs/f2fs/f2fs.h    |   1 +
 fs/f2fs/segment.c |  14 +++++++-
 3 files changed, 74 insertions(+), 42 deletions(-)

-- 
1.9.1

[RFC PATHC 1/2] f2fs: apply write hints to select the type of segments for buffered write

[Hyunchul Lee]

From: Hyunchul Lee <cheol.lee@lge.com>

Write hints helps F2FS to determine which type of segments would be
selected for buffered write.

This patch implements the mapping from write hints to segment types
as shown below.

  hints               segment type
  -----               ------------
  WRITE_LIFE_SHORT    CURSEG_COLD_DATA
  WRITE_LIFE_EXTREME  CURSEG_HOT_DATA
  others              CURSEG_WARM_DATA

the F2FS poliy for hot/cold seperation has precedence over this hints.
And hints are not applied in in-place update.

Signed-off-by: Hyunchul Lee <cheol.lee@lge.com>
---
 fs/f2fs/segment.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index c695ff4..45aef53 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -2258,6 +2258,18 @@ static bool __has_curseg_space(struct f2fs_sb_info *sbi, int type)
 	return false;
 }
 
+int rw_hint_to_seg_type(enum rw_hint hint)
+{
+	switch (hint) {
+	case WRITE_LIFE_SHORT:
+		return CURSEG_HOT_DATA;
+	case WRITE_LIFE_EXTREME:
+		return CURSEG_COLD_DATA;
+	default:
+		return CURSEG_WARM_DATA;
+	}
+}
+
 static int __get_segment_type_2(struct f2fs_io_info *fio)
 {
 	if (fio->type == DATA)
@@ -2292,7 +2304,7 @@ static int __get_segment_type_6(struct f2fs_io_info *fio)
 			return CURSEG_COLD_DATA;
 		if (is_inode_flag_set(inode, FI_HOT_DATA))
 			return CURSEG_HOT_DATA;
-		return CURSEG_WARM_DATA;
+		return rw_hint_to_seg_type(inode->i_write_hint);
 	} else {
 		if (IS_DNODE(fio->page))
 			return is_cold_node(fio->page) ? CURSEG_WARM_NODE :
-- 
1.9.1

[RFC PATHC 2/2] f2fs: apply write hints to select the type of segment for direct write

[Hyunchul Lee]

From: Hyunchul Lee <cheol.lee@lge.com>

Select the type of the segment using write hints, when blocks are
allocated for direct write.

There are unhandled corner cases. Hints are not applied in
in-place update.  And if the blocks of a file is not pre-allocated
because of the invalid user buffer, CURSEG_WARM_DATA segment will
be selected.

Signed-off-by: Hyunchul Lee <cheol.lee@lge.com>
---
 fs/f2fs/data.c | 101 ++++++++++++++++++++++++++++++++++-----------------------
 fs/f2fs/f2fs.h |   1 +
 2 files changed, 61 insertions(+), 41 deletions(-)

diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 36b5352..d06048a 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -783,7 +783,7 @@ struct page *get_new_data_page(struct inode *inode,
 	return page;
 }
 
-static int __allocate_data_block(struct dnode_of_data *dn)
+static int __allocate_data_block(struct dnode_of_data *dn, int seg_type)
 {
 	struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode);
 	struct f2fs_summary sum;
@@ -808,7 +808,7 @@ static int __allocate_data_block(struct dnode_of_data *dn)
 	set_summary(&sum, dn->nid, dn->ofs_in_node, ni.version);
 
 	allocate_data_block(sbi, NULL, dn->data_blkaddr, &dn->data_blkaddr,
-					&sum, CURSEG_WARM_DATA, NULL, false);
+					&sum, seg_type, NULL, false);
 	set_data_blkaddr(dn);
 
 	/* update i_size */
@@ -827,42 +827,6 @@ static inline bool __force_buffered_io(struct inode *inode, int rw)
 			F2FS_I_SB(inode)->s_ndevs);
 }
 
-int f2fs_preallocate_blocks(struct kiocb *iocb, struct iov_iter *from)
-{
-	struct inode *inode = file_inode(iocb->ki_filp);
-	struct f2fs_map_blocks map;
-	int err = 0;
-
-	if (is_inode_flag_set(inode, FI_NO_PREALLOC))
-		return 0;
-
-	map.m_lblk = F2FS_BLK_ALIGN(iocb->ki_pos);
-	map.m_len = F2FS_BYTES_TO_BLK(iocb->ki_pos + iov_iter_count(from));
-	if (map.m_len > map.m_lblk)
-		map.m_len -= map.m_lblk;
-	else
-		map.m_len = 0;
-
-	map.m_next_pgofs = NULL;
-
-	if (iocb->ki_flags & IOCB_DIRECT) {
-		err = f2fs_convert_inline_inode(inode);
-		if (err)
-			return err;
-		return f2fs_map_blocks(inode, &map, 1,
-			__force_buffered_io(inode, WRITE) ?
-				F2FS_GET_BLOCK_PRE_AIO :
-				F2FS_GET_BLOCK_PRE_DIO);
-	}
-	if (iocb->ki_pos + iov_iter_count(from) > MAX_INLINE_DATA(inode)) {
-		err = f2fs_convert_inline_inode(inode);
-		if (err)
-			return err;
-	}
-	if (!f2fs_has_inline_data(inode))
-		return f2fs_map_blocks(inode, &map, 1, F2FS_GET_BLOCK_PRE_AIO);
-	return err;
-}
 
 static inline void __do_map_lock(struct f2fs_sb_info *sbi, int flag, bool lock)
 {
@@ -888,8 +852,8 @@ static inline void __do_map_lock(struct f2fs_sb_info *sbi, int flag, bool lock)
  *     b. do not use extent cache for better performance
  *     c. give the block addresses to blockdev
  */
-int f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map,
-						int create, int flag)
+static int __f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map,
+				int create, int flag, int seg_type)
 {
 	unsigned int maxblocks = map->m_len;
 	struct dnode_of_data dn;
@@ -957,7 +921,12 @@ int f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map,
 					last_ofs_in_node = dn.ofs_in_node;
 				}
 			} else {
-				err = __allocate_data_block(&dn);
+				/* if this inode is marked with FI_NO_PREALLOC,
+				 * @seg_type is NO_CHECK_TYPE
+				 */
+				if (seg_type == NO_CHECK_TYPE)
+					seg_type = CURSEG_WARM_DATA;
+				err = __allocate_data_block(&dn, seg_type);
 				if (!err)
 					set_inode_flag(inode, FI_APPEND_WRITE);
 			}
@@ -1048,6 +1017,51 @@ int f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map,
 	return err;
 }
 
+int f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map,
+						int create, int flag)
+{
+	return __f2fs_map_blocks(inode, map, create, flag, NO_CHECK_TYPE);
+}
+
+int f2fs_preallocate_blocks(struct kiocb *iocb, struct iov_iter *from)
+{
+	struct inode *inode = file_inode(iocb->ki_filp);
+	struct f2fs_map_blocks map;
+	int err = 0;
+
+	if (is_inode_flag_set(inode, FI_NO_PREALLOC))
+		return 0;
+
+	map.m_lblk = F2FS_BLK_ALIGN(iocb->ki_pos);
+	map.m_len = F2FS_BYTES_TO_BLK(iocb->ki_pos + iov_iter_count(from));
+	if (map.m_len > map.m_lblk)
+		map.m_len -= map.m_lblk;
+	else
+		map.m_len = 0;
+
+	map.m_next_pgofs = NULL;
+
+	if (iocb->ki_flags & IOCB_DIRECT) {
+		err = f2fs_convert_inline_inode(inode);
+		if (err)
+			return err;
+		return __f2fs_map_blocks(inode, &map, 1,
+			__force_buffered_io(inode, WRITE) ?
+				F2FS_GET_BLOCK_PRE_AIO :
+				F2FS_GET_BLOCK_PRE_DIO,
+				rw_hint_to_seg_type(iocb->ki_hint));
+	}
+	if (iocb->ki_pos + iov_iter_count(from) > MAX_INLINE_DATA(inode)) {
+		err = f2fs_convert_inline_inode(inode);
+		if (err)
+			return err;
+	}
+	if (!f2fs_has_inline_data(inode))
+		return f2fs_map_blocks(inode, &map, 1, F2FS_GET_BLOCK_PRE_AIO);
+
+	return err;
+}
+
 static int __get_data_block(struct inode *inode, sector_t iblock,
 			struct buffer_head *bh, int create, int flag,
 			pgoff_t *next_pgofs)
@@ -2082,6 +2096,11 @@ static ssize_t f2fs_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
 
 	trace_f2fs_direct_IO_enter(inode, offset, count, rw);
 
+	/* This is for avoiding the situation that the data of a segment is
+	 * passed down to devices with different hints
+	 */
+	iocb->ki_hint = WRITE_LIFE_NOT_SET;
+
 	down_read(&F2FS_I(inode)->dio_rwsem[rw]);
 	err = blockdev_direct_IO(iocb, inode, iter, get_data_block_dio);
 	up_read(&F2FS_I(inode)->dio_rwsem[rw]);
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 4b4a72f..9be5658 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -2562,6 +2562,7 @@ int lookup_journal_in_cursum(struct f2fs_journal *journal, int type,
 void destroy_segment_manager(struct f2fs_sb_info *sbi);
 int __init create_segment_manager_caches(void);
 void destroy_segment_manager_caches(void);
+int rw_hint_to_seg_type(enum rw_hint hint);
 
 /*
  * checkpoint.c
-- 
1.9.1

Re: [RFC PATHC 1/2] f2fs: apply write hints to select the type of segments for buffered write

[Chao Yu]

On 2017/11/9 13:51, Hyunchul Lee wrote:
> From: Hyunchul Lee <cheol.lee@lge.com>
> 
> Write hints helps F2FS to determine which type of segments would be
> selected for buffered write.
> 
> This patch implements the mapping from write hints to segment types
> as shown below.
> 
>   hints               segment type
>   -----               ------------
>   WRITE_LIFE_SHORT    CURSEG_COLD_DATA
>   WRITE_LIFE_EXTREME  CURSEG_HOT_DATA

Should keep consistent with code implementation.

	WRITE_LIFE_SHORT    CURSEG_HOT_DATA
	WRITE_LIFE_EXTREME  CURSEG_COLD_DATA

>   others              CURSEG_WARM_DATA
> 
> the F2FS poliy for hot/cold seperation has precedence over this hints.
> And hints are not applied in in-place update.
> 
> Signed-off-by: Hyunchul Lee <cheol.lee@lge.com>

Reviewed-by: Chao Yu <yuchao0@huawei.com>

Thanks,

> ---
>  fs/f2fs/segment.c | 14 +++++++++++++-
>  1 file changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
> index c695ff4..45aef53 100644
> --- a/fs/f2fs/segment.c
> +++ b/fs/f2fs/segment.c
> @@ -2258,6 +2258,18 @@ static bool __has_curseg_space(struct f2fs_sb_info *sbi, int type)
>  	return false;
>  }
>  
> +int rw_hint_to_seg_type(enum rw_hint hint)
> +{
> +	switch (hint) {
> +	case WRITE_LIFE_SHORT:
> +		return CURSEG_HOT_DATA;
> +	case WRITE_LIFE_EXTREME:
> +		return CURSEG_COLD_DATA;
> +	default:
> +		return CURSEG_WARM_DATA;
> +	}
> +}
> +
>  static int __get_segment_type_2(struct f2fs_io_info *fio)
>  {
>  	if (fio->type == DATA)
> @@ -2292,7 +2304,7 @@ static int __get_segment_type_6(struct f2fs_io_info *fio)
>  			return CURSEG_COLD_DATA;
>  		if (is_inode_flag_set(inode, FI_HOT_DATA))
>  			return CURSEG_HOT_DATA;
> -		return CURSEG_WARM_DATA;
> +		return rw_hint_to_seg_type(inode->i_write_hint);
>  	} else {
>  		if (IS_DNODE(fio->page))
>  			return is_cold_node(fio->page) ? CURSEG_WARM_NODE :
> 

Re: [RFC PATHC 1/2] f2fs: apply write hints to select the type of segments for buffered write

[Jaegeuk Kim]

On 11/09, Chao Yu wrote:
> On 2017/11/9 13:51, Hyunchul Lee wrote:
> > From: Hyunchul Lee <cheol.lee@lge.com>
> > 
> > Write hints helps F2FS to determine which type of segments would be
> > selected for buffered write.
> > 
> > This patch implements the mapping from write hints to segment types
> > as shown below.
> > 
> >   hints               segment type
> >   -----               ------------
> >   WRITE_LIFE_SHORT    CURSEG_COLD_DATA
> >   WRITE_LIFE_EXTREME  CURSEG_HOT_DATA
> 
> Should keep consistent with code implementation.

Merged with this fix. ;)

> 
> 	WRITE_LIFE_SHORT    CURSEG_HOT_DATA
> 	WRITE_LIFE_EXTREME  CURSEG_COLD_DATA
> 
> >   others              CURSEG_WARM_DATA
> > 
> > the F2FS poliy for hot/cold seperation has precedence over this hints.
> > And hints are not applied in in-place update.
> > 
> > Signed-off-by: Hyunchul Lee <cheol.lee@lge.com>
> 
> Reviewed-by: Chao Yu <yuchao0@huawei.com>
> 
> Thanks,
> 
> > ---
> >  fs/f2fs/segment.c | 14 +++++++++++++-
> >  1 file changed, 13 insertions(+), 1 deletion(-)
> > 
> > diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
> > index c695ff4..45aef53 100644
> > --- a/fs/f2fs/segment.c
> > +++ b/fs/f2fs/segment.c
> > @@ -2258,6 +2258,18 @@ static bool __has_curseg_space(struct f2fs_sb_info *sbi, int type)
> >  	return false;
> >  }
> >  
> > +int rw_hint_to_seg_type(enum rw_hint hint)
> > +{
> > +	switch (hint) {
> > +	case WRITE_LIFE_SHORT:
> > +		return CURSEG_HOT_DATA;
> > +	case WRITE_LIFE_EXTREME:
> > +		return CURSEG_COLD_DATA;
> > +	default:
> > +		return CURSEG_WARM_DATA;
> > +	}
> > +}
> > +
> >  static int __get_segment_type_2(struct f2fs_io_info *fio)
> >  {
> >  	if (fio->type == DATA)
> > @@ -2292,7 +2304,7 @@ static int __get_segment_type_6(struct f2fs_io_info *fio)
> >  			return CURSEG_COLD_DATA;
> >  		if (is_inode_flag_set(inode, FI_HOT_DATA))
> >  			return CURSEG_HOT_DATA;
> > -		return CURSEG_WARM_DATA;
> > +		return rw_hint_to_seg_type(inode->i_write_hint);
> >  	} else {
> >  		if (IS_DNODE(fio->page))
> >  			return is_cold_node(fio->page) ? CURSEG_WARM_NODE :
> > 

Re: [RFC PATCH 0/2] apply write hints to select the type of segments

[Christoph Hellwig]

Next time please coordinate this with the block list and Jens, who
actually wrote the patch.

>   hints                     segment type
>   -----                     ------------
>   WRITE_LIFE_SHORT          CURSEG_COLD_DATA
>   WRITE_LIFE_EXTREME        CURSEG_HOT_DATA
>   others                    CURSEG_WARM_DATA

Normally cold data is data with a long lifetime, and extreme is colder
than cold, so there seems to be some mismatch here.

Re: [RFC PATCH 0/2] apply write hints to select the type of segments

[Jaegeuk Kim]

On 11/17, Christoph Hellwig wrote:
> 
> Next time please coordinate this with the block list and Jens, who
> actually wrote the patch.

Got it.

> 
> >   hints                     segment type
> >   -----                     ------------
> >   WRITE_LIFE_SHORT          CURSEG_COLD_DATA
> >   WRITE_LIFE_EXTREME        CURSEG_HOT_DATA
> >   others                    CURSEG_WARM_DATA
> 
> Normally cold data is data with a long lifetime, and extreme is colder
> than cold, so there seems to be some mismatch here.

It was wrong description and I fixed it which matches to implementation.

The below description was merged:

WRITE_LIFE_SHORT          CURSEG_HOT_DATA
WRITE_LIFE_EXTREME        CURSEG_COLD_DATA
others                    CURSEG_WARM_DATA