Linux-f2fs-devel Archive on lore.kernel.org
 help / color / Atom feed
* [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?
@ 2020-07-10  3:33 lampahome
  2020-07-10  3:41 ` Jaegeuk Kim
  0 siblings, 1 reply; 7+ messages in thread
From: lampahome @ 2020-07-10  3:33 UTC (permalink / raw)
  To: linux-f2fs-devel

I create 3 dd job write 1GB file with directIO and blocksize is 2MB
equal to segment size.

But it seems the each IO of three job's physical block address maybe
in different segment.

If I can get full segment for one directIO, I thought performance will be up.

Is that possible?


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?
  2020-07-10  3:33 [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB? lampahome
@ 2020-07-10  3:41 ` Jaegeuk Kim
       [not found]   ` <CAB3eZfv5k7cwFzMAOda7+cgHJzatRRhwyEg-8U9yt8TZ5s0=ww@mail.gmail.com>
       [not found]   ` <CAB3eZfuaBhGPHDCt8v_9MrFW1byni17wzLtwG-Ny40q6jj9jkw@mail.gmail.com>
  0 siblings, 2 replies; 7+ messages in thread
From: Jaegeuk Kim @ 2020-07-10  3:41 UTC (permalink / raw)
  To: lampahome; +Cc: linux-f2fs-devel

On 07/10, lampahome wrote:
> I create 3 dd job write 1GB file with directIO and blocksize is 2MB
> equal to segment size.
> 
> But it seems the each IO of three job's physical block address maybe
> in different segment.
> 
> If I can get full segment for one directIO, I thought performance will be up.
> 
> Is that possible?

How about using file pinning?

1. fd = create()
2. ioctl(fd, F2FS_IOC_SET_PIN_FILE)
3. fallocate(1GB)
 -> will allocate 2MB segments to the 1GB space
4. write(DIRECT_IO)
 -> will do in-place updates directly

You can use f2fs_io built by f2fs-tools for #2.
# f2fs_io pinfile set [file_path]

> 
> 
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?
       [not found]   ` <CAB3eZfv5k7cwFzMAOda7+cgHJzatRRhwyEg-8U9yt8TZ5s0=ww@mail.gmail.com>
@ 2020-07-10  4:25     ` Jaegeuk Kim
       [not found]       ` <CAB3eZfvgEsv_T0AC6imnmFvTw3-hE6-BOd1TPqqqP5RS9bHLQA@mail.gmail.com>
  0 siblings, 1 reply; 7+ messages in thread
From: Jaegeuk Kim @ 2020-07-10  4:25 UTC (permalink / raw)
  To: lampahome; +Cc: linux-f2fs-devel

On 07/10, lampahome wrote:
> > How about using file pinning?
> 
> I mean if IO blocksize or file size is multiple times of segment size
> e.g. segment size = 2MB, and directIO blocksize is always 2MB.
> I hope the 512 sub-blocks(4K blocksize) of each directIO will be in the
> same segment.
> 
> e.g.  segment size = 2MB, I want to create a 2MB file.
> I hope the 512 sub-blocks(4K blocksize) will be in the same segment.
> 
> PIN file before fallocate will let allocated blocks in a whole full
> segment?
> Doc only wrote:
> 
> > However, once F2FS receives ioctl(fd, F2FS_IOC_SET_PIN_FILE) in prior to
> > fallocate(fd, DEFAULT_MODE), it allocates on-disk blocks addressess having
> > zero or random data.

Previously, fallocate doesn't allocate block addressess, but pinned file allows
to allocate them aligned to 2MB-sized segment.

> 
> 
> Or do I misunderstand?
> 
> thanks


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?
       [not found]       ` <CAB3eZfvgEsv_T0AC6imnmFvTw3-hE6-BOd1TPqqqP5RS9bHLQA@mail.gmail.com>
@ 2020-07-10  5:14         ` Jaegeuk Kim
       [not found]           ` <CAB3eZfsm5Z7pvD1enSioHDUys7+SekkNOscW11WDjDPWF=ngog@mail.gmail.com>
  0 siblings, 1 reply; 7+ messages in thread
From: Jaegeuk Kim @ 2020-07-10  5:14 UTC (permalink / raw)
  To: lampahome; +Cc: linux-f2fs-devel

On 07/10, lampahome wrote:
> >
> > Previously, fallocate doesn't allocate block addressess, but pinned file
> > allows
> > to allocate them aligned to 2MB-sized segment.
> >
> > So I create a pin file such as 10MB and it needs 5 segments.
> Maybe the 5 segments are not continuous, but I can make sure the all
> 4k-blocks of the file are all in full segments.
> Am I right?

I think so. You can check fibmap to see the block addresses. :)



_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?
       [not found]           ` <CAB3eZfsm5Z7pvD1enSioHDUys7+SekkNOscW11WDjDPWF=ngog@mail.gmail.com>
@ 2020-07-10  6:28             ` Jaegeuk Kim
  0 siblings, 0 replies; 7+ messages in thread
From: Jaegeuk Kim @ 2020-07-10  6:28 UTC (permalink / raw)
  To: lampahome; +Cc: linux-f2fs-devel

On 07/10, lampahome wrote:
> >
> > I think so. You can check fibmap to see the block addresses. :)
> >
> > Execuse me, I'm a new to f2fs, how to check fibmap?
> You mean f2fs-tools?

It's generic ioctl.

hdparm --fibmap [file]

Or,

fibmap.f2fs [file]

> 
> thanks


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?
       [not found]   ` <CAB3eZfuaBhGPHDCt8v_9MrFW1byni17wzLtwG-Ny40q6jj9jkw@mail.gmail.com>
@ 2020-07-13 17:57     ` Jaegeuk Kim
  2020-07-15 19:18     ` Jaegeuk Kim
  1 sibling, 0 replies; 7+ messages in thread
From: Jaegeuk Kim @ 2020-07-13 17:57 UTC (permalink / raw)
  To: lampahome; +Cc: linux-f2fs-devel

On 07/10, lampahome wrote:
> >
> > 1. fd = create()
> > 2. ioctl(fd, F2FS_IOC_SET_PIN_FILE)
> > 3. fallocate(1GB)
> >  -> will allocate 2MB segments to the 1GB space
> >
> >
> I tried but it shows ioctl failed: Bad address.

Hmm, can I ask which kernel you're using?

Thanks,

> 
> Code below:
> #define _GNU_SOURCE
> #include <stdlib.h>
> #include <stdio.h>
> #include <sys/ioctl.h>
> #include <fcntl.h>
> #include <unistd.h>
> #include <string.h>
> #include <sys/param.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <stdint.h>
> 
> typedef uint32_t __u32;
> 
> #define F2FS_IOCTL_MAGIC                0xf5
> #define F2FS_IOC_SET_PIN_FILE           _IOW(F2FS_IOCTL_MAGIC, 13, __u32)
> 
> int main() {
>  char prefix[] = "/mnt/f2fsdir/";
>  char path[5][10] = {"ggg1","ggg2","ggg3","ggg4","ggg5"};
>  int i, num=1, ret;
>  int fd;
>  FILE *fp;
>  off_t off = 0;
>  off_t size = 512;
>  char buf[256];
>  mode_t mode = S_IWUSR | S_IWGRP | S_IWOTH | S_IRUSR | S_IRGRP | S_IROTH;
> 
> 
>  for (int j=0;j<num;j++) {
>   memset(buf, 0, sizeof(buf));
>   snprintf(buf, sizeof(buf), "%s%s", prefix, path[j]);
>   fd = creat(buf, mode);
>   if (fd < 0)
>    perror("open fd failed");
> 
>   ret = ioctl(fd, F2FS_IOC_SET_PIN_FILE);
>   if (ret < 0)
>    perror("ioctl failed");
> 
>   if (fallocate(fd, 0, off, size)<0)
>    perror("could not allocate");
> 
>   close(fd);
>  }
> 
>  return 0;
> }


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB?
       [not found]   ` <CAB3eZfuaBhGPHDCt8v_9MrFW1byni17wzLtwG-Ny40q6jj9jkw@mail.gmail.com>
  2020-07-13 17:57     ` Jaegeuk Kim
@ 2020-07-15 19:18     ` Jaegeuk Kim
  1 sibling, 0 replies; 7+ messages in thread
From: Jaegeuk Kim @ 2020-07-15 19:18 UTC (permalink / raw)
  To: lampahome; +Cc: linux-f2fs-devel

On 07/10, lampahome wrote:
> >
> > 1. fd = create()
> > 2. ioctl(fd, F2FS_IOC_SET_PIN_FILE)
> > 3. fallocate(1GB)
> >  -> will allocate 2MB segments to the 1GB space
> >
> >
> I tried but it shows ioctl failed: Bad address.
> 
> Code below:
> #define _GNU_SOURCE
> #include <stdlib.h>
> #include <stdio.h>
> #include <sys/ioctl.h>
> #include <fcntl.h>
> #include <unistd.h>
> #include <string.h>
> #include <sys/param.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <stdint.h>
> 
> typedef uint32_t __u32;
> 
> #define F2FS_IOCTL_MAGIC                0xf5
> #define F2FS_IOC_SET_PIN_FILE           _IOW(F2FS_IOCTL_MAGIC, 13, __u32)
> 
> int main() {
>  char prefix[] = "/mnt/f2fsdir/";
>  char path[5][10] = {"ggg1","ggg2","ggg3","ggg4","ggg5"};
>  int i, num=1, ret;
>  int fd;
>  FILE *fp;
>  off_t off = 0;
>  off_t size = 512;
>  char buf[256];
>  mode_t mode = S_IWUSR | S_IWGRP | S_IWOTH | S_IRUSR | S_IRGRP | S_IROTH;
> 
> 
>  for (int j=0;j<num;j++) {
>   memset(buf, 0, sizeof(buf));
>   snprintf(buf, sizeof(buf), "%s%s", prefix, path[j]);
>   fd = creat(buf, mode);
>   if (fd < 0)
>    perror("open fd failed");
> 
>   ret = ioctl(fd, F2FS_IOC_SET_PIN_FILE);

This needs to be like this.
https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-tools.git/tree/tools/f2fs_io/f2fs_io.c#n357

>   if (ret < 0)
>    perror("ioctl failed");
> 
>   if (fallocate(fd, 0, off, size)<0)
>    perror("could not allocate");
> 
>   close(fd);
>  }
> 
>  return 0;
> }


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, back to index

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-10  3:33 [f2fs-dev] possible to allocate a full segment for a direct IO with blocksize 2MB? lampahome
2020-07-10  3:41 ` Jaegeuk Kim
     [not found]   ` <CAB3eZfv5k7cwFzMAOda7+cgHJzatRRhwyEg-8U9yt8TZ5s0=ww@mail.gmail.com>
2020-07-10  4:25     ` Jaegeuk Kim
     [not found]       ` <CAB3eZfvgEsv_T0AC6imnmFvTw3-hE6-BOd1TPqqqP5RS9bHLQA@mail.gmail.com>
2020-07-10  5:14         ` Jaegeuk Kim
     [not found]           ` <CAB3eZfsm5Z7pvD1enSioHDUys7+SekkNOscW11WDjDPWF=ngog@mail.gmail.com>
2020-07-10  6:28             ` Jaegeuk Kim
     [not found]   ` <CAB3eZfuaBhGPHDCt8v_9MrFW1byni17wzLtwG-Ny40q6jj9jkw@mail.gmail.com>
2020-07-13 17:57     ` Jaegeuk Kim
2020-07-15 19:18     ` Jaegeuk Kim

Linux-f2fs-devel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-f2fs-devel/0 linux-f2fs-devel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-f2fs-devel linux-f2fs-devel/ https://lore.kernel.org/linux-f2fs-devel \
		linux-f2fs-devel@lists.sourceforge.net
	public-inbox-index linux-f2fs-devel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/net.sourceforge.lists.linux-f2fs-devel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git