[f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?

[f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?

[lampahome]

I create 3 dd job write 1GB file with directIO and blocksize is 2MB
equal to segment size.

But it seems the each IO of three job's physical block address maybe
in different segment.

If I can get full segment for one directIO, I thought performance will be up.

Is that possible?


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?

[Jaegeuk Kim]

On 07/10, lampahome wrote:
> I create 3 dd job write 1GB file with directIO and blocksize is 2MB
> equal to segment size.
> 
> But it seems the each IO of three job's physical block address maybe
> in different segment.
> 
> If I can get full segment for one directIO, I thought performance will be up.
> 
> Is that possible?

How about using file pinning?

1. fd = create()
2. ioctl(fd, F2FS_IOC_SET_PIN_FILE)
3. fallocate(1GB)
 -> will allocate 2MB segments to the 1GB space
4. write(DIRECT_IO)
 -> will do in-place updates directly

You can use f2fs_io built by f2fs-tools for #2.
# f2fs_io pinfile set [file_path]

> 
> 
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?

[Jaegeuk Kim]

On 07/10, lampahome wrote:
> > How about using file pinning?
> 
> I mean if IO blocksize or file size is multiple times of segment size
> e.g. segment size = 2MB, and directIO blocksize is always 2MB.
> I hope the 512 sub-blocks(4K blocksize) of each directIO will be in the
> same segment.
> 
> e.g.  segment size = 2MB, I want to create a 2MB file.
> I hope the 512 sub-blocks(4K blocksize) will be in the same segment.
> 
> PIN file before fallocate will let allocated blocks in a whole full
> segment?
> Doc only wrote:
> 
> > However, once F2FS receives ioctl(fd, F2FS_IOC_SET_PIN_FILE) in prior to
> > fallocate(fd, DEFAULT_MODE), it allocates on-disk blocks addressess having
> > zero or random data.

Previously, fallocate doesn't allocate block addressess, but pinned file allows
to allocate them aligned to 2MB-sized segment.

> 
> 
> Or do I misunderstand?
> 
> thanks


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?

[Jaegeuk Kim]

On 07/10, lampahome wrote:
> >
> > Previously, fallocate doesn't allocate block addressess, but pinned file
> > allows
> > to allocate them aligned to 2MB-sized segment.
> >
> > So I create a pin file such as 10MB and it needs 5 segments.
> Maybe the 5 segments are not continuous, but I can make sure the all
> 4k-blocks of the file are all in full segments.
> Am I right?

I think so. You can check fibmap to see the block addresses. :)



_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?

[Jaegeuk Kim]

On 07/10, lampahome wrote:
> >
> > I think so. You can check fibmap to see the block addresses. :)
> >
> > Execuse me, I'm a new to f2fs, how to check fibmap?
> You mean f2fs-tools?

It's generic ioctl.

hdparm --fibmap [file]

Or,

fibmap.f2fs [file]

> 
> thanks


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?

[Jaegeuk Kim]

On 07/10, lampahome wrote:
> >
> > 1. fd = create()
> > 2. ioctl(fd, F2FS_IOC_SET_PIN_FILE)
> > 3. fallocate(1GB)
> >  -> will allocate 2MB segments to the 1GB space
> >
> >
> I tried but it shows ioctl failed: Bad address.

Hmm, can I ask which kernel you're using?

Thanks,

> 
> Code below:
> #define _GNU_SOURCE
> #include <stdlib.h>
> #include <stdio.h>
> #include <sys/ioctl.h>
> #include <fcntl.h>
> #include <unistd.h>
> #include <string.h>
> #include <sys/param.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <stdint.h>
> 
> typedef uint32_t __u32;
> 
> #define F2FS_IOCTL_MAGIC                0xf5
> #define F2FS_IOC_SET_PIN_FILE           _IOW(F2FS_IOCTL_MAGIC, 13, __u32)
> 
> int main() {
>  char prefix[] = "/mnt/f2fsdir/";
>  char path[5][10] = {"ggg1","ggg2","ggg3","ggg4","ggg5"};
>  int i, num=1, ret;
>  int fd;
>  FILE *fp;
>  off_t off = 0;
>  off_t size = 512;
>  char buf[256];
>  mode_t mode = S_IWUSR | S_IWGRP | S_IWOTH | S_IRUSR | S_IRGRP | S_IROTH;
> 
> 
>  for (int j=0;j<num;j++) {
>   memset(buf, 0, sizeof(buf));
>   snprintf(buf, sizeof(buf), "%s%s", prefix, path[j]);
>   fd = creat(buf, mode);
>   if (fd < 0)
>    perror("open fd failed");
> 
>   ret = ioctl(fd, F2FS_IOC_SET_PIN_FILE);
>   if (ret < 0)
>    perror("ioctl failed");
> 
>   if (fallocate(fd, 0, off, size)<0)
>    perror("could not allocate");
> 
>   close(fd);
>  }
> 
>  return 0;
> }


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?

[Jaegeuk Kim]

On 07/10, lampahome wrote:
> >
> > 1. fd = create()
> > 2. ioctl(fd, F2FS_IOC_SET_PIN_FILE)
> > 3. fallocate(1GB)
> >  -> will allocate 2MB segments to the 1GB space
> >
> >
> I tried but it shows ioctl failed: Bad address.
> 
> Code below:
> #define _GNU_SOURCE
> #include <stdlib.h>
> #include <stdio.h>
> #include <sys/ioctl.h>
> #include <fcntl.h>
> #include <unistd.h>
> #include <string.h>
> #include <sys/param.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <stdint.h>
> 
> typedef uint32_t __u32;
> 
> #define F2FS_IOCTL_MAGIC                0xf5
> #define F2FS_IOC_SET_PIN_FILE           _IOW(F2FS_IOCTL_MAGIC, 13, __u32)
> 
> int main() {
>  char prefix[] = "/mnt/f2fsdir/";
>  char path[5][10] = {"ggg1","ggg2","ggg3","ggg4","ggg5"};
>  int i, num=1, ret;
>  int fd;
>  FILE *fp;
>  off_t off = 0;
>  off_t size = 512;
>  char buf[256];
>  mode_t mode = S_IWUSR | S_IWGRP | S_IWOTH | S_IRUSR | S_IRGRP | S_IROTH;
> 
> 
>  for (int j=0;j<num;j++) {
>   memset(buf, 0, sizeof(buf));
>   snprintf(buf, sizeof(buf), "%s%s", prefix, path[j]);
>   fd = creat(buf, mode);
>   if (fd < 0)
>    perror("open fd failed");
> 
>   ret = ioctl(fd, F2FS_IOC_SET_PIN_FILE);

This needs to be like this.
https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-tools.git/tree/tools/f2fs_io/f2fs_io.c#n357

>   if (ret < 0)
>    perror("ioctl failed");
> 
>   if (fallocate(fd, 0, off, size)<0)
>    perror("could not allocate");
> 
>   close(fd);
>  }
> 
>  return 0;
> }


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel