Re: internel implemention of file operation

Re: internel implemention of file operation

[dspjmt]

i suggest you do a little before you ask any questions.
first, we never load a whole file into memory, second, it is hardly relevent to vm struct 

Sent from my HTC

----- Reply message -----
From: "horseriver" <horserivers@gmail.com>
To: <kernelnewbies@kernelnewbies.org>
Subject: internel implemention of file operation
Date: Thu, Jan 3, 2013 2:59 pm


hi:

  when one file is opened , does its  data be put into memory ? and all operation on this file

  will be implemented by operation on its mapping memory area ?

thanks!

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130202/0d662590/attachment.html 

internel implemention of file operation

[Rajat Sharma]

> Default read/write inerfaces does not move file's data to process address
space ?
Yes it does, in either case, user space memory has to be in process address
space. But Difference is in the access pattern. With read/write, you demand
for it explicitly through a system call, hence application is more involved
here. While for mmap access, all transfer happens in application-unaware
mechanism, with page-fault handlers inside kernel. Application just access
mapped buffers like memory array and magic happens inside kernel as you
keep on accessing bytes.

Best way to find out difference is to try out writing a simple mmap program.

-Rajat


On Fri, Jan 11, 2013 at 3:31 AM, horseriver <horserivers@gmail.com> wrote:

> On Fri, Jan 11, 2013 at 12:39:26PM +0530, Rajat Sharma wrote:
>
> > Default read/write inerfaces are better suited for sequential read/write
> > within your program. Although you can seek to any location within the
> file,
> > you still have overhead to issue system calls to get data. However mmap
> > allows you to map a section of file into program address space.
>
>   Default read/write inerfaces does not move file's data to process
> address space ?
>
>   when  r/w a file descript which returnd by open() , how do the file data
> move from one place to another place ?
>
>   For each time the write function being  called  , will kernel  call
> filesystem's driver's write  to respond  ??
>   In my opinion,kernel will passed a  buffer's head address  which is
> passed form user-layer into driver,then driver will fill this buffer with
> file's
>   data which is got by filesystem's read operation ?
>
>   Am I right?
>
>   Thanks!
> >
> >
> > -Rajat
> >
> >
> > On Fri, Jan 11, 2013 at 2:44 AM, horseriver <horserivers@gmail.com>
> wrote:
> >
> > > hi:
> > >
> > >   these two wayes of operating one file :
> > >
> > >   1.use open/write interface call .
> > >
> > >   2.mmap this file into memory , then access this memory area and do
> r/w .
> > >
> > >   what is the essential difference between this teo wayes?
> > >
> > > thanks!
> > >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130111/5ffa8257/attachment-0001.html 

internel implemention of file operation

[Rajat Sharma]

Default read/write inerfaces are better suited for sequential read/write
within your program. Although you can seek to any location within the file,
you still have overhead to issue system calls to get data. However mmap
allows you to map a section of file into program address space. Now if your
access patter is rather random, modifying few bytes here and there, but at
random offset. You just get continous memory array, which is much easier
than issuing read and write at different file offsets.

A most common and mandatory use case of mmap is in mapping executable
binary program image, and libraries into process address spaces. Access
pattern for a program is not sequential, you can have multiple jump (if,
else, for loop), so it is better suited with mmap. It is read only and
private memory mapping, Any modifications you do will create a COW page
which if private to your process, so that is another advantage of mmap
which is completely transparent to user mode. If your filesystem does not
support, this basic mmap mode, you can not execute a binary file stored in
this filesystem, unless you copy it to some other filesystem which does.

Apart from this, read more about mmap from UTLK book.

-Rajat


On Fri, Jan 11, 2013 at 2:44 AM, horseriver <horserivers@gmail.com> wrote:

> hi:
>
>   these two wayes of operating one file :
>
>   1.use open/write interface call .
>
>   2.mmap this file into memory , then access this memory area and do r/w .
>
>   what is the essential difference between this teo wayes?
>
> thanks!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130111/5e5810b2/attachment.html 

internel implemention of file operation

[horseriver]

On Fri, Jan 11, 2013 at 12:39:26PM +0530, Rajat Sharma wrote:

> Default read/write inerfaces are better suited for sequential read/write
> within your program. Although you can seek to any location within the file,
> you still have overhead to issue system calls to get data. However mmap
> allows you to map a section of file into program address space. 

  Default read/write inerfaces does not move file's data to process address space ?

  when  r/w a file descript which returnd by open() , how do the file data move from one place to another place ?

  For each time the write function being  called  , will kernel  call filesystem's driver's write  to respond  ??
  In my opinion,kernel will passed a  buffer's head address  which is passed form user-layer into driver,then driver will fill this buffer with file's
  data which is got by filesystem's read operation ?

  Am I right?   
  
  Thanks! 
> 
> 
> -Rajat
> 
> 
> On Fri, Jan 11, 2013 at 2:44 AM, horseriver <horserivers@gmail.com> wrote:
> 
> > hi:
> >
> >   these two wayes of operating one file :
> >
> >   1.use open/write interface call .
> >
> >   2.mmap this file into memory , then access this memory area and do r/w .
> >
> >   what is the essential difference between this teo wayes?
> >
> > thanks!
> >

internel implemention of file operation

[horseriver]

hi:

  these two wayes of operating one file :

  1.use open/write interface call .

  2.mmap this file into memory , then access this memory area and do r/w . 

  what is the essential difference between this teo wayes?

thanks!

internel implemention of file operation

[Rajat Sharma]

> in this vm_operations_struct , there are open/close functions ,  are
there necessary  relations between file operations and this struct ?
well not really for open/close of vm_ops are of interest to filesystems,
but page fault handler and making page writable is where filesystem come
into picture. Have a look at ext4_file_vm_ops, it implements operations of
interest.

static const struct vm_operations_struct ext4_file_vm_ops = {
        .fault          = filemap_fault,
        .page_mkwrite   = ext4_page_mkwrite,
}

Note that only filesystem knows how to fill up this page.


On Thu, Jan 3, 2013 at 3:39 PM, horseriver <horserivers@gmail.com> wrote:

> On Thu, Jan 03, 2013 at 01:16:06PM +0530, Rajat Sharma wrote:
>
> > > will it be maped with vm_area struct ?
> > Yes if it is accessed via mmap system call.
>
> you know that , in the struct vm_area_struct,there is a struct
> vm_operations_struct * vm_ops;
>
> in this vm_operations_struct , there are open/close functions ,  are there
> necessary  relations between file
>
> operations and this struct ?
>
> thanks!
>
> >
> > > what is the relation between page-cache and file operation?
> > file operations for data access like read/write will look into page-cache
> > first before going to disk.
> >
> > -Rajat
> >
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130103/2fa3c832/attachment.html 

internel implemention of file operation

[horseriver]

On Thu, Jan 03, 2013 at 01:16:06PM +0530, Rajat Sharma wrote:

> > will it be maped with vm_area struct ?
> Yes if it is accessed via mmap system call.

you know that , in the struct vm_area_struct,there is a struct vm_operations_struct * vm_ops;

in this vm_operations_struct , there are open/close functions ,  are there necessary  relations between file 

operations and this struct ?

thanks!

> 
> > what is the relation between page-cache and file operation?
> file operations for data access like read/write will look into page-cache
> first before going to disk.
> 
> -Rajat
> 

internel implemention of file operation

[Rajat Sharma]

unfortunately these are not the topic to digest in a mail, I recommend you
reading UTLK book (http://shop.oreilly.com/product/9780596005658.do).

Still...

> will it be maped with vm_area struct ?
Yes if it is accessed via mmap system call.

> what is the relation between page-cache and file operation?
file operations for data access like read/write will look into page-cache
first before going to disk.

-Rajat


On Thu, Jan 3, 2013 at 1:01 PM, horseriver <horserivers@gmail.com> wrote:

> On Thu, Jan 03, 2013 at 12:48:01PM +0530, Rajat Sharma wrote:
> > Never heard of page-cache?
>
> will it be maped with vm_area struct ?
> what is the relation between page-cache and file operation?
>
> thanks!
>
> >
> >
> > On Thu, Jan 3, 2013 at 12:29 PM, horseriver <horserivers@gmail.com>
> wrote:
> >
> > > hi:
> > >
> > >   when one file is opened , does its  data be put into memory ? and all
> > > operation on this file
> > >
> > >   will be implemented by operation on its mapping memory area ?
> > >
> > > thanks!
> > >
> > > _______________________________________________
> > > Kernelnewbies mailing list
> > > Kernelnewbies at kernelnewbies.org
> > > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> > >
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130103/de3ffb1a/attachment.html 

internel implemention of file operation

[horseriver]

On Thu, Jan 03, 2013 at 12:48:01PM +0530, Rajat Sharma wrote:
> Never heard of page-cache?

will it be maped with vm_area struct ?
what is the relation between page-cache and file operation?

thanks!

> 
> 
> On Thu, Jan 3, 2013 at 12:29 PM, horseriver <horserivers@gmail.com> wrote:
> 
> > hi:
> >
> >   when one file is opened , does its  data be put into memory ? and all
> > operation on this file
> >
> >   will be implemented by operation on its mapping memory area ?
> >
> > thanks!
> >
> > _______________________________________________
> > Kernelnewbies mailing list
> > Kernelnewbies at kernelnewbies.org
> > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> >

internel implemention of file operation

[Rajat Sharma]

Never heard of page-cache?


On Thu, Jan 3, 2013 at 12:29 PM, horseriver <horserivers@gmail.com> wrote:

> hi:
>
>   when one file is opened , does its  data be put into memory ? and all
> operation on this file
>
>   will be implemented by operation on its mapping memory area ?
>
> thanks!
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130103/9972b726/attachment.html 

internel implemention of file operation

[horseriver]

hi:

  when one file is opened , does its  data be put into memory ? and all operation on this file

  will be implemented by operation on its mapping memory area ?

thanks!