* Fwd: Question about kernel interfaces [not found] <CADxyqMFnDnFNvTL7Yuzd-1WP9RRddFYAd7UVXmLT2SOMRmOx9g@mail.gmail.com> @ 2015-01-07 12:16 ` Siddhartha De 2015-01-08 13:48 ` One Thousand Gnomes 0 siblings, 1 reply; 18+ messages in thread From: Siddhartha De @ 2015-01-07 12:16 UTC (permalink / raw) To: linux-kernel I have already asked this question on the kernelnewbies IRC but I wanted a bit of feedback from the Linux gurus ... Is sysfs 'supposed' to be a read only where various kernel parameters are exposed to be read ? Or are they 'supposed' to be writable too ? Are all drivers in future required to expose interfaces in sysfs ? If the answer to the above is true , then would we be really requiring IOCTL calls at all anymore ... because all that we would need to do to make a driver do something is to change values in some of the files in sysfs ... for eg to call a ioctl called DO_SOMETHING on driver D1 we would be doing : echo 1> /sysfs/D1/IOCTLS/DO_SOMETHING ( or something like that ... :) ) Are there any plans of exposing the kernel api ( that is syscalls and libc ) as sysfs files ... for eg echo 1> /sysfs/libc/get_system_time cat /sysfs/libc/results/system_time Would it be a good idea to do this ? How about exposing core system calls , libc and any new libraries over http/over the network ? This would make calling ioctls and really low level stuff from high level languages like Java a trivial process . For eg right now if I wanted to call some function say foo () in library my_library.so in Java I would have to take the JNI/JNA route ... doable but not exactly 'easy' ( same for other high level languages ) ... whereas in the new way all I would have to do is send a request to say :- http://localhost:7000/my_library/foo?arg1=something,arg2=something ( Just wanted a frank discussion on this idea .... ) Thanking you in anticipation , Yours sincerely , Kernel newbie ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Question about kernel interfaces 2015-01-07 12:16 ` Fwd: Question about kernel interfaces Siddhartha De @ 2015-01-08 13:48 ` One Thousand Gnomes 2015-01-09 11:48 ` Siddhartha De 0 siblings, 1 reply; 18+ messages in thread From: One Thousand Gnomes @ 2015-01-08 13:48 UTC (permalink / raw) To: Siddhartha De; +Cc: linux-kernel > Is sysfs 'supposed' to be a read only where various kernel parameters > are exposed to be read ? Or are they 'supposed' to be writable too ? You can create either > Are all drivers in future required to expose interfaces in sysfs ? Only if they need one (some will appear automatically as you enable power management and the like). > If the answer to the above is true , then would we be really requiring > IOCTL calls at all anymore ... because all that we would need to do to > make a driver do something is to change values in some of the files in > sysfs ... for eg to call a ioctl called DO_SOMETHING on driver D1 we > would be doing : > > echo 1> /sysfs/D1/IOCTLS/DO_SOMETHING ( or something like that ... :) ) Try implementing a few this way and you'll discover you can't. ioctls happen on file handles which are tied to instances of an object not to the driver. ioctl also allows read/write pairs so can be serializing and sequenced in a way sysfs can't. > Are there any plans of exposing the kernel api ( that is syscalls and > libc ) as sysfs files ... for eg > > echo 1> /sysfs/libc/get_system_time > cat /sysfs/libc/results/system_time Time is an example where you could implement it via sysfs. It's an unusual case in that its global and stateless. > Would it be a good idea to do this ? No, probably not. > How about exposing core system calls , libc and any new libraries over > http/over the network ? This would make calling ioctls and really low > level stuff from high level languages like Java a trivial process . > For eg right now if I wanted to call some function say foo () in > library my_library.so in Java I would have to take the JNI/JNA route > ... doable but not exactly 'easy' ( same for other high level > languages ) ... whereas in the new way all I would have to do is send > a request to say :- > http://localhost:7000/my_library/foo?arg1=something,arg2=something > ( Just wanted a frank discussion on this idea .... ) Except that you've also got to transfer the data and the authentication back and forth. You need much more than sysfs for this, you need some way to pass messages securely, and to do work on a remote node on your behalf. You need some way to locate a resource. It's not entirely impossible to make syscalls remotable. It's been done to some extent at least two ways. Read up on plan 9 (which did try and take Unix much more networked and within the classic tradition of Unix as a small clean OS), and perhaps also on MOSIX, which took a very different approach to running on a cluster of boxes as one instance. Another odd example to look at is Minix. Minix treats all syscalls internally as messages between user processes and services like the file system. Minix never really made use of it, but it does mean its much easier to do remote syscall models in Minix than Linux. (in part I suspect this comes from Tanenbaum's Amoeba and similar work). I'm not sure I'd pick Java for this either - Java is very weak in this area as it tends to expose notions of locality and what is and isn't remote when doing method invocations. Languages like smalltalk on the other hand got this right many years before. Alan ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Question about kernel interfaces 2015-01-08 13:48 ` One Thousand Gnomes @ 2015-01-09 11:48 ` Siddhartha De 2015-01-09 12:20 ` Siddhartha De 2015-01-14 16:33 ` One Thousand Gnomes 0 siblings, 2 replies; 18+ messages in thread From: Siddhartha De @ 2015-01-09 11:48 UTC (permalink / raw) To: One Thousand Gnomes; +Cc: linux-kernel Hi , Thanks for your reply . Really appreciate your taking time out to answer this ... 1> > Are all drivers in future required to expose interfaces in sysfs ? > Only if they need one (some will appear automatically as you enable power management and the like). Just wanted to point out something :- Quoting from makelinux website :- http://www.makelinux.net/books/lkd2/ch17lev1sec8 "The sysfs filesystem is currently the place for implementing functionality previously reserved for ioctl() calls on device nodes or the procfs filesystem. These days, the chic thing to do is implement such functionality as sysfs attributes in the appropriate directory. For example, instead of a new ioctl() on a device node, add a sysfs attribute in the driver's sysfs directory. Such an approach avoids the type-unsafe use of obscure ioctl() arguments and the haphazard mess of /proc. " Quoting from the busybox mailing list archives :- Hi vda, all, ISTR that sysfs was the long-term plan of the kernel folks and that they mean to completely(?) drop ioctl support. In what timeframe should we migrate to "pure" sysfs and drop using any ioctls (if a sysfs interface exists)? cheers, Bernhard Quote from livejournal http://zaitcev.livejournal.com/84788.html sysfs is the new ioctl With all these spread all over the net , any newbie like myself is bound to get confused ... add to that I just saw some old mails from Linus where he blasts the ioctl way of doing things ( http://yarchive.net/comp/linux/ioctl.html ) :) 2> It's not entirely impossible to make syscalls remotable. It's been done to some extent at least two ways. Read up on plan 9 (which did try and take Unix much more networked and within the classic tradition of Unix as a small clean OS), Thanks for the advice , I did read up a bit on Plan 9 , but from what I understood it was trying to abstract the network to the user ... a bit different from what my confusion is .... but a great resource nevertheless 3> >I'm not sure I'd pick Java for this either - Java is very weak in this area as it tends to expose notions of locality and what is and isn't remote when doing method invocations. Languages like smalltalk on the other hand got this right many years before. Yeah ... the reason I mentioned Java is because it represents a typical high level language commonly used by today's programmers ... the problem with all these languages is that while they do a fine job when writing web based applications they really make it difficult to access low level system calls ( for eg talking to the driver and stuff ) . Moreover when a library is written for say Linux or Windows the library becomes tied to that particular language ( the one it is written in ) ... for eg let us say we write a new library in C for playing mp3 's in Linux ... if we are to write a user space program and want to access the library 's functionality we have to write the userspace program in C ... if we write it in any other language ( say Java ) we have to use other methods like Java Native Interface ( JNI ... or more recently JNA ) to access this ... not a good way . What I am saying is let us expose the library over http/any other network protocol ... if any other language needs to access the library's functionality it simply makes a request like :- http://localhost:7000/libmp3/foo?arg1=something&arg2=something This way we avoid a lot of rework since once a library is written it's functionality is exposed to all languages ( Java / PHP / Python ) . Taking this idea further , could we also expose driver IOCTL's over the network ? For eg to open the cd rom drive we would be calling the following url :- (assuming device is /dev/cdrom ) http://localhost:7000/IOCTL/IOCTL_EJECT?device=/dev/cdrom The listener program ( our interpreter listening on port 7000 ) would be taking this url call and doing the actual IOCTL call file_handle= open ( /dev/cdrom ) call_IOCTL ( file_handle,IOCTL_EJECT, &some_structure ) The idea of exposing all these interfaces as url's is not to make them accessible over the network ( although this could be a possibility ...) but simply to make them accessible from any user space program in a language neutral way ... Thanks Kernel Newbie :) On Thu, Jan 8, 2015 at 7:18 PM, One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote: >> Is sysfs 'supposed' to be a read only where various kernel parameters >> are exposed to be read ? Or are they 'supposed' to be writable too ? > > You can create either > >> Are all drivers in future required to expose interfaces in sysfs ? > > Only if they need one (some will appear automatically as you enable power > management and the like). > >> If the answer to the above is true , then would we be really requiring >> IOCTL calls at all anymore ... because all that we would need to do to >> make a driver do something is to change values in some of the files in >> sysfs ... for eg to call a ioctl called DO_SOMETHING on driver D1 we >> would be doing : >> >> echo 1> /sysfs/D1/IOCTLS/DO_SOMETHING ( or something like that ... :) ) > > Try implementing a few this way and you'll discover you can't. ioctls > happen on file handles which are tied to instances of an object not to > the driver. ioctl also allows read/write pairs so can be serializing and > sequenced in a way sysfs can't. > >> Are there any plans of exposing the kernel api ( that is syscalls and >> libc ) as sysfs files ... for eg >> >> echo 1> /sysfs/libc/get_system_time >> cat /sysfs/libc/results/system_time > > Time is an example where you could implement it via sysfs. It's an > unusual case in that its global and stateless. > >> Would it be a good idea to do this ? > > No, probably not. > >> How about exposing core system calls , libc and any new libraries over >> http/over the network ? This would make calling ioctls and really low >> level stuff from high level languages like Java a trivial process . >> For eg right now if I wanted to call some function say foo () in >> library my_library.so in Java I would have to take the JNI/JNA route >> ... doable but not exactly 'easy' ( same for other high level >> languages ) ... whereas in the new way all I would have to do is send >> a request to say :- >> http://localhost:7000/my_library/foo?arg1=something,arg2=something >> ( Just wanted a frank discussion on this idea .... ) > > Except that you've also got to transfer the data and the authentication > back and forth. You need much more than sysfs for this, you need some way > to pass messages securely, and to do work on a remote node on your > behalf. You need some way to locate a resource. > > It's not entirely impossible to make syscalls remotable. It's been done to > some extent at least two ways. Read up on plan 9 (which did try and take > Unix much more networked and within the classic tradition of Unix as a > small clean OS), and perhaps also on MOSIX, which took a very different > approach to running on a cluster of boxes as one instance. Another odd > example to look at is Minix. Minix treats all syscalls internally as > messages between user processes and services like the file system. Minix > never really made use of it, but it does mean its much easier to do > remote syscall models in Minix than Linux. (in part I suspect this comes > from Tanenbaum's Amoeba and similar work). > > I'm not sure I'd pick Java for this either - Java is very weak in this > area as it tends to expose notions of locality and what is and isn't > remote when doing method invocations. Languages like smalltalk on the > other hand got this right many years before. > > Alan > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Fwd: Question about kernel interfaces 2015-01-09 11:48 ` Siddhartha De @ 2015-01-09 12:20 ` Siddhartha De 2015-01-14 16:33 ` One Thousand Gnomes 1 sibling, 0 replies; 18+ messages in thread From: Siddhartha De @ 2015-01-09 12:20 UTC (permalink / raw) To: linux-kernel, Siddhartha De, kernelnewbies Hi , Dear Kernel Newbies , I have had some discussions regarding this on the IRC but I thought it would be a better idea to post it to the mailing list as well .... just in case someone came up with any ideas/comments/suggestions later .... PS :- I am marking a copy to the main linux-kernel list so that any discussions automatically gets forwarded there ... Thanks , Kernel Newbie :) ---------- Forwarded message ---------- From: Siddhartha De <siddhartha.de87@gmail.com> Date: Fri, Jan 9, 2015 at 5:18 PM Subject: Re: Question about kernel interfaces To: One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> Cc: linux-kernel <linux-kernel@vger.kernel.org> Hi , Thanks for your reply . Really appreciate your taking time out to answer this ... 1> > Are all drivers in future required to expose interfaces in sysfs ? > Only if they need one (some will appear automatically as you enable power management and the like). Just wanted to point out something :- Quoting from makelinux website :- http://www.makelinux.net/books/lkd2/ch17lev1sec8 "The sysfs filesystem is currently the place for implementing functionality previously reserved for ioctl() calls on device nodes or the procfs filesystem. These days, the chic thing to do is implement such functionality as sysfs attributes in the appropriate directory. For example, instead of a new ioctl() on a device node, add a sysfs attribute in the driver's sysfs directory. Such an approach avoids the type-unsafe use of obscure ioctl() arguments and the haphazard mess of /proc. " Quoting from the busybox mailing list archives :- Hi vda, all, ISTR that sysfs was the long-term plan of the kernel folks and that they mean to completely(?) drop ioctl support. In what timeframe should we migrate to "pure" sysfs and drop using any ioctls (if a sysfs interface exists)? cheers, Bernhard Quote from livejournal http://zaitcev.livejournal.com/84788.html sysfs is the new ioctl With all these spread all over the net , any newbie like myself is bound to get confused ... add to that I just saw some old mails from Linus where he blasts the ioctl way of doing things ( http://yarchive.net/comp/linux/ioctl.html ) :) 2> It's not entirely impossible to make syscalls remotable. It's been done to some extent at least two ways. Read up on plan 9 (which did try and take Unix much more networked and within the classic tradition of Unix as a small clean OS), Thanks for the advice , I did read up a bit on Plan 9 , but from what I understood it was trying to abstract the network to the user ... a bit different from what my confusion is .... but a great resource nevertheless 3> >I'm not sure I'd pick Java for this either - Java is very weak in this area as it tends to expose notions of locality and what is and isn't remote when doing method invocations. Languages like smalltalk on the other hand got this right many years before. Yeah ... the reason I mentioned Java is because it represents a typical high level language commonly used by today's programmers ... the problem with all these languages is that while they do a fine job when writing web based applications they really make it difficult to access low level system calls ( for eg talking to the driver and stuff ) . Moreover when a library is written for say Linux or Windows the library becomes tied to that particular language ( the one it is written in ) ... for eg let us say we write a new library in C for playing mp3 's in Linux ... if we are to write a user space program and want to access the library 's functionality we have to write the userspace program in C ... if we write it in any other language ( say Java ) we have to use other methods like Java Native Interface ( JNI ... or more recently JNA ) to access this ... not a good way . What I am saying is let us expose the library over http/any other network protocol ... if any other language needs to access the library's functionality it simply makes a request like :- http://localhost:7000/libmp3/foo?arg1=something&arg2=something This way we avoid a lot of rework since once a library is written it's functionality is exposed to all languages ( Java / PHP / Python ) . Taking this idea further , could we also expose driver IOCTL's over the network ? For eg to open the cd rom drive we would be calling the following url :- (assuming device is /dev/cdrom ) http://localhost:7000/IOCTL/IOCTL_EJECT?device=/dev/cdrom The listener program ( our interpreter listening on port 7000 ) would be taking this url call and doing the actual IOCTL call file_handle= open ( /dev/cdrom ) call_IOCTL ( file_handle,IOCTL_EJECT, &some_structure ) The idea of exposing all these interfaces as url's is not to make them accessible over the network ( although this could be a possibility ...) but simply to make them accessible from any user space program in a language neutral way ... Thanks Kernel Newbie :) On Thu, Jan 8, 2015 at 7:18 PM, One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote: >> Is sysfs 'supposed' to be a read only where various kernel parameters >> are exposed to be read ? Or are they 'supposed' to be writable too ? > > You can create either > >> Are all drivers in future required to expose interfaces in sysfs ? > > Only if they need one (some will appear automatically as you enable power > management and the like). > >> If the answer to the above is true , then would we be really requiring >> IOCTL calls at all anymore ... because all that we would need to do to >> make a driver do something is to change values in some of the files in >> sysfs ... for eg to call a ioctl called DO_SOMETHING on driver D1 we >> would be doing : >> >> echo 1> /sysfs/D1/IOCTLS/DO_SOMETHING ( or something like that ... :) ) > > Try implementing a few this way and you'll discover you can't. ioctls > happen on file handles which are tied to instances of an object not to > the driver. ioctl also allows read/write pairs so can be serializing and > sequenced in a way sysfs can't. > >> Are there any plans of exposing the kernel api ( that is syscalls and >> libc ) as sysfs files ... for eg >> >> echo 1> /sysfs/libc/get_system_time >> cat /sysfs/libc/results/system_time > > Time is an example where you could implement it via sysfs. It's an > unusual case in that its global and stateless. > >> Would it be a good idea to do this ? > > No, probably not. > >> How about exposing core system calls , libc and any new libraries over >> http/over the network ? This would make calling ioctls and really low >> level stuff from high level languages like Java a trivial process . >> For eg right now if I wanted to call some function say foo () in >> library my_library.so in Java I would have to take the JNI/JNA route >> ... doable but not exactly 'easy' ( same for other high level >> languages ) ... whereas in the new way all I would have to do is send >> a request to say :- >> http://localhost:7000/my_library/foo?arg1=something,arg2=something >> ( Just wanted a frank discussion on this idea .... ) > > Except that you've also got to transfer the data and the authentication > back and forth. You need much more than sysfs for this, you need some way > to pass messages securely, and to do work on a remote node on your > behalf. You need some way to locate a resource. > > It's not entirely impossible to make syscalls remotable. It's been done to > some extent at least two ways. Read up on plan 9 (which did try and take > Unix much more networked and within the classic tradition of Unix as a > small clean OS), and perhaps also on MOSIX, which took a very different > approach to running on a cluster of boxes as one instance. Another odd > example to look at is Minix. Minix treats all syscalls internally as > messages between user processes and services like the file system. Minix > never really made use of it, but it does mean its much easier to do > remote syscall models in Minix than Linux. (in part I suspect this comes > from Tanenbaum's Amoeba and similar work). > > I'm not sure I'd pick Java for this either - Java is very weak in this > area as it tends to expose notions of locality and what is and isn't > remote when doing method invocations. Languages like smalltalk on the > other hand got this right many years before. > > Alan > On Wednesday Jan 7th Siddhartha De (siddhartha.de87@gmail.com ) wrote :- I have already asked this question on the kernelnewbies IRC but I wanted a bit of feedback from the Linux gurus ... Is sysfs 'supposed' to be a read only where various kernel parameters are exposed to be read ? Or are they 'supposed' to be writable too ? Are all drivers in future required to expose interfaces in sysfs ? If the answer to the above is true , then would we be really requiring IOCTL calls at all anymore ... because all that we would need to do to make a driver do something is to change values in some of the files in sysfs ... for eg to call a ioctl called DO_SOMETHING on driver D1 we would be doing : echo 1> /sysfs/D1/IOCTLS/DO_SOMETHING ( or something like that ... :) ) Are there any plans of exposing the kernel api ( that is syscalls and libc ) as sysfs files ... for eg echo 1> /sysfs/libc/get_system_time cat /sysfs/libc/results/system_time Would it be a good idea to do this ? How about exposing core system calls , libc and any new libraries over http/over the network ? This would make calling ioctls and really low level stuff from high level languages like Java a trivial process . For eg right now if I wanted to call some function say foo () in library my_library.so in Java I would have to take the JNI/JNA route ... doable but not exactly 'easy' ( same for other high level languages ) ... whereas in the new way all I would have to do is send a request to say :- http://localhost:7000/my_library/foo?arg1=something,arg2=something ( Just wanted a frank discussion on this idea .... ) Thanking you in anticipation , Yours sincerely , Kernel newbie ^ permalink raw reply [flat|nested] 18+ messages in thread
* Fwd: Question about kernel interfaces @ 2015-01-09 12:20 ` Siddhartha De 0 siblings, 0 replies; 18+ messages in thread From: Siddhartha De @ 2015-01-09 12:20 UTC (permalink / raw) To: kernelnewbies Hi , Dear Kernel Newbies , I have had some discussions regarding this on the IRC but I thought it would be a better idea to post it to the mailing list as well .... just in case someone came up with any ideas/comments/suggestions later .... PS :- I am marking a copy to the main linux-kernel list so that any discussions automatically gets forwarded there ... Thanks , Kernel Newbie :) ---------- Forwarded message ---------- From: Siddhartha De <siddhartha.de87@gmail.com> Date: Fri, Jan 9, 2015 at 5:18 PM Subject: Re: Question about kernel interfaces To: One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> Cc: linux-kernel <linux-kernel@vger.kernel.org> Hi , Thanks for your reply . Really appreciate your taking time out to answer this ... 1> > Are all drivers in future required to expose interfaces in sysfs ? > Only if they need one (some will appear automatically as you enable power management and the like). Just wanted to point out something :- Quoting from makelinux website :- http://www.makelinux.net/books/lkd2/ch17lev1sec8 "The sysfs filesystem is currently the place for implementing functionality previously reserved for ioctl() calls on device nodes or the procfs filesystem. These days, the chic thing to do is implement such functionality as sysfs attributes in the appropriate directory. For example, instead of a new ioctl() on a device node, add a sysfs attribute in the driver's sysfs directory. Such an approach avoids the type-unsafe use of obscure ioctl() arguments and the haphazard mess of /proc. " Quoting from the busybox mailing list archives :- Hi vda, all, ISTR that sysfs was the long-term plan of the kernel folks and that they mean to completely(?) drop ioctl support. In what timeframe should we migrate to "pure" sysfs and drop using any ioctls (if a sysfs interface exists)? cheers, Bernhard Quote from livejournal http://zaitcev.livejournal.com/84788.html sysfs is the new ioctl With all these spread all over the net , any newbie like myself is bound to get confused ... add to that I just saw some old mails from Linus where he blasts the ioctl way of doing things ( http://yarchive.net/comp/linux/ioctl.html ) :) 2> It's not entirely impossible to make syscalls remotable. It's been done to some extent at least two ways. Read up on plan 9 (which did try and take Unix much more networked and within the classic tradition of Unix as a small clean OS), Thanks for the advice , I did read up a bit on Plan 9 , but from what I understood it was trying to abstract the network to the user ... a bit different from what my confusion is .... but a great resource nevertheless 3> >I'm not sure I'd pick Java for this either - Java is very weak in this area as it tends to expose notions of locality and what is and isn't remote when doing method invocations. Languages like smalltalk on the other hand got this right many years before. Yeah ... the reason I mentioned Java is because it represents a typical high level language commonly used by today's programmers ... the problem with all these languages is that while they do a fine job when writing web based applications they really make it difficult to access low level system calls ( for eg talking to the driver and stuff ) . Moreover when a library is written for say Linux or Windows the library becomes tied to that particular language ( the one it is written in ) ... for eg let us say we write a new library in C for playing mp3 's in Linux ... if we are to write a user space program and want to access the library 's functionality we have to write the userspace program in C ... if we write it in any other language ( say Java ) we have to use other methods like Java Native Interface ( JNI ... or more recently JNA ) to access this ... not a good way . What I am saying is let us expose the library over http/any other network protocol ... if any other language needs to access the library's functionality it simply makes a request like :- http://localhost:7000/libmp3/foo?arg1=something&arg2=something This way we avoid a lot of rework since once a library is written it's functionality is exposed to all languages ( Java / PHP / Python ) . Taking this idea further , could we also expose driver IOCTL's over the network ? For eg to open the cd rom drive we would be calling the following url :- (assuming device is /dev/cdrom ) http://localhost:7000/IOCTL/IOCTL_EJECT?device=/dev/cdrom The listener program ( our interpreter listening on port 7000 ) would be taking this url call and doing the actual IOCTL call file_handle= open ( /dev/cdrom ) call_IOCTL ( file_handle,IOCTL_EJECT, &some_structure ) The idea of exposing all these interfaces as url's is not to make them accessible over the network ( although this could be a possibility ...) but simply to make them accessible from any user space program in a language neutral way ... Thanks Kernel Newbie :) On Thu, Jan 8, 2015 at 7:18 PM, One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote: >> Is sysfs 'supposed' to be a read only where various kernel parameters >> are exposed to be read ? Or are they 'supposed' to be writable too ? > > You can create either > >> Are all drivers in future required to expose interfaces in sysfs ? > > Only if they need one (some will appear automatically as you enable power > management and the like). > >> If the answer to the above is true , then would we be really requiring >> IOCTL calls at all anymore ... because all that we would need to do to >> make a driver do something is to change values in some of the files in >> sysfs ... for eg to call a ioctl called DO_SOMETHING on driver D1 we >> would be doing : >> >> echo 1> /sysfs/D1/IOCTLS/DO_SOMETHING ( or something like that ... :) ) > > Try implementing a few this way and you'll discover you can't. ioctls > happen on file handles which are tied to instances of an object not to > the driver. ioctl also allows read/write pairs so can be serializing and > sequenced in a way sysfs can't. > >> Are there any plans of exposing the kernel api ( that is syscalls and >> libc ) as sysfs files ... for eg >> >> echo 1> /sysfs/libc/get_system_time >> cat /sysfs/libc/results/system_time > > Time is an example where you could implement it via sysfs. It's an > unusual case in that its global and stateless. > >> Would it be a good idea to do this ? > > No, probably not. > >> How about exposing core system calls , libc and any new libraries over >> http/over the network ? This would make calling ioctls and really low >> level stuff from high level languages like Java a trivial process . >> For eg right now if I wanted to call some function say foo () in >> library my_library.so in Java I would have to take the JNI/JNA route >> ... doable but not exactly 'easy' ( same for other high level >> languages ) ... whereas in the new way all I would have to do is send >> a request to say :- >> http://localhost:7000/my_library/foo?arg1=something,arg2=something >> ( Just wanted a frank discussion on this idea .... ) > > Except that you've also got to transfer the data and the authentication > back and forth. You need much more than sysfs for this, you need some way > to pass messages securely, and to do work on a remote node on your > behalf. You need some way to locate a resource. > > It's not entirely impossible to make syscalls remotable. It's been done to > some extent at least two ways. Read up on plan 9 (which did try and take > Unix much more networked and within the classic tradition of Unix as a > small clean OS), and perhaps also on MOSIX, which took a very different > approach to running on a cluster of boxes as one instance. Another odd > example to look at is Minix. Minix treats all syscalls internally as > messages between user processes and services like the file system. Minix > never really made use of it, but it does mean its much easier to do > remote syscall models in Minix than Linux. (in part I suspect this comes > from Tanenbaum's Amoeba and similar work). > > I'm not sure I'd pick Java for this either - Java is very weak in this > area as it tends to expose notions of locality and what is and isn't > remote when doing method invocations. Languages like smalltalk on the > other hand got this right many years before. > > Alan > On Wednesday Jan 7th Siddhartha De (siddhartha.de87 at gmail.com ) wrote :- I have already asked this question on the kernelnewbies IRC but I wanted a bit of feedback from the Linux gurus ... Is sysfs 'supposed' to be a read only where various kernel parameters are exposed to be read ? Or are they 'supposed' to be writable too ? Are all drivers in future required to expose interfaces in sysfs ? If the answer to the above is true , then would we be really requiring IOCTL calls at all anymore ... because all that we would need to do to make a driver do something is to change values in some of the files in sysfs ... for eg to call a ioctl called DO_SOMETHING on driver D1 we would be doing : echo 1> /sysfs/D1/IOCTLS/DO_SOMETHING ( or something like that ... :) ) Are there any plans of exposing the kernel api ( that is syscalls and libc ) as sysfs files ... for eg echo 1> /sysfs/libc/get_system_time cat /sysfs/libc/results/system_time Would it be a good idea to do this ? How about exposing core system calls , libc and any new libraries over http/over the network ? This would make calling ioctls and really low level stuff from high level languages like Java a trivial process . For eg right now if I wanted to call some function say foo () in library my_library.so in Java I would have to take the JNI/JNA route ... doable but not exactly 'easy' ( same for other high level languages ) ... whereas in the new way all I would have to do is send a request to say :- http://localhost:7000/my_library/foo?arg1=something,arg2=something ( Just wanted a frank discussion on this idea .... ) Thanking you in anticipation , Yours sincerely , Kernel newbie ^ permalink raw reply [flat|nested] 18+ messages in thread
* Fwd: Question about kernel interfaces 2015-01-09 12:20 ` Siddhartha De @ 2015-01-09 12:30 ` Siddhartha De -1 siblings, 0 replies; 18+ messages in thread From: Siddhartha De @ 2015-01-09 12:30 UTC (permalink / raw) To: linux-kernel, Siddhartha De, kernelnewbies Looks like the last message bounced ... Thanks , Kernel Newbie :) ---------- Forwarded message ---------- From: Siddhartha De <siddhartha.de87@gmail.com> Date: Fri, Jan 9, 2015 at 5:50 PM Subject: Fwd: Question about kernel interfaces To: linux-kernel <linux-kernel@vger.kernel.org>, Siddhartha De <siddhartha.de87@gmail.com>, kernelnewbies@kernelnewbies.org Hi , Dear Kernel Newbies , I have had some discussions regarding this on the IRC but I thought it would be a better idea to post it to the mailing list as well .... just in case someone came up with any ideas/comments/suggestions later .... PS :- I am marking a copy to the main linux-kernel list so that any discussions automatically gets forwarded there ... Thanks , Kernel Newbie :) ---------- Forwarded message ---------- From: Siddhartha De <siddhartha.de87@gmail.com> Date: Fri, Jan 9, 2015 at 5:18 PM Subject: Re: Question about kernel interfaces To: One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> Cc: linux-kernel <linux-kernel@vger.kernel.org> Hi , Thanks for your reply . Really appreciate your taking time out to answer this ... 1> > Are all drivers in future required to expose interfaces in sysfs ? > Only if they need one (some will appear automatically as you enable power management and the like). Just wanted to point out something :- Quoting from makelinux website :- http://www.makelinux.net/books/lkd2/ch17lev1sec8 "The sysfs filesystem is currently the place for implementing functionality previously reserved for ioctl() calls on device nodes or the procfs filesystem. These days, the chic thing to do is implement such functionality as sysfs attributes in the appropriate directory. For example, instead of a new ioctl() on a device node, add a sysfs attribute in the driver's sysfs directory. Such an approach avoids the type-unsafe use of obscure ioctl() arguments and the haphazard mess of /proc. " Quoting from the busybox mailing list archives :- Hi vda, all, ISTR that sysfs was the long-term plan of the kernel folks and that they mean to completely(?) drop ioctl support. In what timeframe should we migrate to "pure" sysfs and drop using any ioctls (if a sysfs interface exists)? cheers, Bernhard Quote from livejournal http://zaitcev.livejournal.com/84788.html sysfs is the new ioctl With all these spread all over the net , any newbie like myself is bound to get confused ... add to that I just saw some old mails from Linus where he blasts the ioctl way of doing things ( http://yarchive.net/comp/linux/ioctl.html ) :) 2> It's not entirely impossible to make syscalls remotable. It's been done to some extent at least two ways. Read up on plan 9 (which did try and take Unix much more networked and within the classic tradition of Unix as a small clean OS), Thanks for the advice , I did read up a bit on Plan 9 , but from what I understood it was trying to abstract the network to the user ... a bit different from what my confusion is .... but a great resource nevertheless 3> >I'm not sure I'd pick Java for this either - Java is very weak in this area as it tends to expose notions of locality and what is and isn't remote when doing method invocations. Languages like smalltalk on the other hand got this right many years before. Yeah ... the reason I mentioned Java is because it represents a typical high level language commonly used by today's programmers ... the problem with all these languages is that while they do a fine job when writing web based applications they really make it difficult to access low level system calls ( for eg talking to the driver and stuff ) . Moreover when a library is written for say Linux or Windows the library becomes tied to that particular language ( the one it is written in ) ... for eg let us say we write a new library in C for playing mp3 's in Linux ... if we are to write a user space program and want to access the library 's functionality we have to write the userspace program in C ... if we write it in any other language ( say Java ) we have to use other methods like Java Native Interface ( JNI ... or more recently JNA ) to access this ... not a good way . What I am saying is let us expose the library over http/any other network protocol ... if any other language needs to access the library's functionality it simply makes a request like :- http://localhost:7000/libmp3/foo?arg1=something&arg2=something This way we avoid a lot of rework since once a library is written it's functionality is exposed to all languages ( Java / PHP / Python ) . Taking this idea further , could we also expose driver IOCTL's over the network ? For eg to open the cd rom drive we would be calling the following url :- (assuming device is /dev/cdrom ) http://localhost:7000/IOCTL/IOCTL_EJECT?device=/dev/cdrom The listener program ( our interpreter listening on port 7000 ) would be taking this url call and doing the actual IOCTL call file_handle= open ( /dev/cdrom ) call_IOCTL ( file_handle,IOCTL_EJECT, &some_structure ) The idea of exposing all these interfaces as url's is not to make them accessible over the network ( although this could be a possibility ...) but simply to make them accessible from any user space program in a language neutral way ... Thanks Kernel Newbie :) On Thu, Jan 8, 2015 at 7:18 PM, One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote: >> Is sysfs 'supposed' to be a read only where various kernel parameters >> are exposed to be read ? Or are they 'supposed' to be writable too ? > > You can create either > >> Are all drivers in future required to expose interfaces in sysfs ? > > Only if they need one (some will appear automatically as you enable power > management and the like). > >> If the answer to the above is true , then would we be really requiring >> IOCTL calls at all anymore ... because all that we would need to do to >> make a driver do something is to change values in some of the files in >> sysfs ... for eg to call a ioctl called DO_SOMETHING on driver D1 we >> would be doing : >> >> echo 1> /sysfs/D1/IOCTLS/DO_SOMETHING ( or something like that ... :) ) > > Try implementing a few this way and you'll discover you can't. ioctls > happen on file handles which are tied to instances of an object not to > the driver. ioctl also allows read/write pairs so can be serializing and > sequenced in a way sysfs can't. > >> Are there any plans of exposing the kernel api ( that is syscalls and >> libc ) as sysfs files ... for eg >> >> echo 1> /sysfs/libc/get_system_time >> cat /sysfs/libc/results/system_time > > Time is an example where you could implement it via sysfs. It's an > unusual case in that its global and stateless. > >> Would it be a good idea to do this ? > > No, probably not. > >> How about exposing core system calls , libc and any new libraries over >> http/over the network ? This would make calling ioctls and really low >> level stuff from high level languages like Java a trivial process . >> For eg right now if I wanted to call some function say foo () in >> library my_library.so in Java I would have to take the JNI/JNA route >> ... doable but not exactly 'easy' ( same for other high level >> languages ) ... whereas in the new way all I would have to do is send >> a request to say :- >> http://localhost:7000/my_library/foo?arg1=something,arg2=something >> ( Just wanted a frank discussion on this idea .... ) > > Except that you've also got to transfer the data and the authentication > back and forth. You need much more than sysfs for this, you need some way > to pass messages securely, and to do work on a remote node on your > behalf. You need some way to locate a resource. > > It's not entirely impossible to make syscalls remotable. It's been done to > some extent at least two ways. Read up on plan 9 (which did try and take > Unix much more networked and within the classic tradition of Unix as a > small clean OS), and perhaps also on MOSIX, which took a very different > approach to running on a cluster of boxes as one instance. Another odd > example to look at is Minix. Minix treats all syscalls internally as > messages between user processes and services like the file system. Minix > never really made use of it, but it does mean its much easier to do > remote syscall models in Minix than Linux. (in part I suspect this comes > from Tanenbaum's Amoeba and similar work). > > I'm not sure I'd pick Java for this either - Java is very weak in this > area as it tends to expose notions of locality and what is and isn't > remote when doing method invocations. Languages like smalltalk on the > other hand got this right many years before. > > Alan > On Wednesday Jan 7th Siddhartha De (siddhartha.de87@gmail.com ) wrote :- I have already asked this question on the kernelnewbies IRC but I wanted a bit of feedback from the Linux gurus ... Is sysfs 'supposed' to be a read only where various kernel parameters are exposed to be read ? Or are they 'supposed' to be writable too ? Are all drivers in future required to expose interfaces in sysfs ? If the answer to the above is true , then would we be really requiring IOCTL calls at all anymore ... because all that we would need to do to make a driver do something is to change values in some of the files in sysfs ... for eg to call a ioctl called DO_SOMETHING on driver D1 we would be doing : echo 1> /sysfs/D1/IOCTLS/DO_SOMETHING ( or something like that ... :) ) Are there any plans of exposing the kernel api ( that is syscalls and libc ) as sysfs files ... for eg echo 1> /sysfs/libc/get_system_time cat /sysfs/libc/results/system_time Would it be a good idea to do this ? How about exposing core system calls , libc and any new libraries over http/over the network ? This would make calling ioctls and really low level stuff from high level languages like Java a trivial process . For eg right now if I wanted to call some function say foo () in library my_library.so in Java I would have to take the JNI/JNA route ... doable but not exactly 'easy' ( same for other high level languages ) ... whereas in the new way all I would have to do is send a request to say :- http://localhost:7000/my_library/foo?arg1=something,arg2=something ( Just wanted a frank discussion on this idea .... ) Thanking you in anticipation , Yours sincerely , Kernel newbie ^ permalink raw reply [flat|nested] 18+ messages in thread
* Fwd: Question about kernel interfaces @ 2015-01-09 12:30 ` Siddhartha De 0 siblings, 0 replies; 18+ messages in thread From: Siddhartha De @ 2015-01-09 12:30 UTC (permalink / raw) To: kernelnewbies Looks like the last message bounced ... Thanks , Kernel Newbie :) ---------- Forwarded message ---------- From: Siddhartha De <siddhartha.de87@gmail.com> Date: Fri, Jan 9, 2015 at 5:50 PM Subject: Fwd: Question about kernel interfaces To: linux-kernel <linux-kernel@vger.kernel.org>, Siddhartha De <siddhartha.de87@gmail.com>, kernelnewbies at kernelnewbies.org Hi , Dear Kernel Newbies , I have had some discussions regarding this on the IRC but I thought it would be a better idea to post it to the mailing list as well .... just in case someone came up with any ideas/comments/suggestions later .... PS :- I am marking a copy to the main linux-kernel list so that any discussions automatically gets forwarded there ... Thanks , Kernel Newbie :) ---------- Forwarded message ---------- From: Siddhartha De <siddhartha.de87@gmail.com> Date: Fri, Jan 9, 2015 at 5:18 PM Subject: Re: Question about kernel interfaces To: One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> Cc: linux-kernel <linux-kernel@vger.kernel.org> Hi , Thanks for your reply . Really appreciate your taking time out to answer this ... 1> > Are all drivers in future required to expose interfaces in sysfs ? > Only if they need one (some will appear automatically as you enable power management and the like). Just wanted to point out something :- Quoting from makelinux website :- http://www.makelinux.net/books/lkd2/ch17lev1sec8 "The sysfs filesystem is currently the place for implementing functionality previously reserved for ioctl() calls on device nodes or the procfs filesystem. These days, the chic thing to do is implement such functionality as sysfs attributes in the appropriate directory. For example, instead of a new ioctl() on a device node, add a sysfs attribute in the driver's sysfs directory. Such an approach avoids the type-unsafe use of obscure ioctl() arguments and the haphazard mess of /proc. " Quoting from the busybox mailing list archives :- Hi vda, all, ISTR that sysfs was the long-term plan of the kernel folks and that they mean to completely(?) drop ioctl support. In what timeframe should we migrate to "pure" sysfs and drop using any ioctls (if a sysfs interface exists)? cheers, Bernhard Quote from livejournal http://zaitcev.livejournal.com/84788.html sysfs is the new ioctl With all these spread all over the net , any newbie like myself is bound to get confused ... add to that I just saw some old mails from Linus where he blasts the ioctl way of doing things ( http://yarchive.net/comp/linux/ioctl.html ) :) 2> It's not entirely impossible to make syscalls remotable. It's been done to some extent at least two ways. Read up on plan 9 (which did try and take Unix much more networked and within the classic tradition of Unix as a small clean OS), Thanks for the advice , I did read up a bit on Plan 9 , but from what I understood it was trying to abstract the network to the user ... a bit different from what my confusion is .... but a great resource nevertheless 3> >I'm not sure I'd pick Java for this either - Java is very weak in this area as it tends to expose notions of locality and what is and isn't remote when doing method invocations. Languages like smalltalk on the other hand got this right many years before. Yeah ... the reason I mentioned Java is because it represents a typical high level language commonly used by today's programmers ... the problem with all these languages is that while they do a fine job when writing web based applications they really make it difficult to access low level system calls ( for eg talking to the driver and stuff ) . Moreover when a library is written for say Linux or Windows the library becomes tied to that particular language ( the one it is written in ) ... for eg let us say we write a new library in C for playing mp3 's in Linux ... if we are to write a user space program and want to access the library 's functionality we have to write the userspace program in C ... if we write it in any other language ( say Java ) we have to use other methods like Java Native Interface ( JNI ... or more recently JNA ) to access this ... not a good way . What I am saying is let us expose the library over http/any other network protocol ... if any other language needs to access the library's functionality it simply makes a request like :- http://localhost:7000/libmp3/foo?arg1=something&arg2=something This way we avoid a lot of rework since once a library is written it's functionality is exposed to all languages ( Java / PHP / Python ) . Taking this idea further , could we also expose driver IOCTL's over the network ? For eg to open the cd rom drive we would be calling the following url :- (assuming device is /dev/cdrom ) http://localhost:7000/IOCTL/IOCTL_EJECT?device=/dev/cdrom The listener program ( our interpreter listening on port 7000 ) would be taking this url call and doing the actual IOCTL call file_handle= open ( /dev/cdrom ) call_IOCTL ( file_handle,IOCTL_EJECT, &some_structure ) The idea of exposing all these interfaces as url's is not to make them accessible over the network ( although this could be a possibility ...) but simply to make them accessible from any user space program in a language neutral way ... Thanks Kernel Newbie :) On Thu, Jan 8, 2015 at 7:18 PM, One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote: >> Is sysfs 'supposed' to be a read only where various kernel parameters >> are exposed to be read ? Or are they 'supposed' to be writable too ? > > You can create either > >> Are all drivers in future required to expose interfaces in sysfs ? > > Only if they need one (some will appear automatically as you enable power > management and the like). > >> If the answer to the above is true , then would we be really requiring >> IOCTL calls at all anymore ... because all that we would need to do to >> make a driver do something is to change values in some of the files in >> sysfs ... for eg to call a ioctl called DO_SOMETHING on driver D1 we >> would be doing : >> >> echo 1> /sysfs/D1/IOCTLS/DO_SOMETHING ( or something like that ... :) ) > > Try implementing a few this way and you'll discover you can't. ioctls > happen on file handles which are tied to instances of an object not to > the driver. ioctl also allows read/write pairs so can be serializing and > sequenced in a way sysfs can't. > >> Are there any plans of exposing the kernel api ( that is syscalls and >> libc ) as sysfs files ... for eg >> >> echo 1> /sysfs/libc/get_system_time >> cat /sysfs/libc/results/system_time > > Time is an example where you could implement it via sysfs. It's an > unusual case in that its global and stateless. > >> Would it be a good idea to do this ? > > No, probably not. > >> How about exposing core system calls , libc and any new libraries over >> http/over the network ? This would make calling ioctls and really low >> level stuff from high level languages like Java a trivial process . >> For eg right now if I wanted to call some function say foo () in >> library my_library.so in Java I would have to take the JNI/JNA route >> ... doable but not exactly 'easy' ( same for other high level >> languages ) ... whereas in the new way all I would have to do is send >> a request to say :- >> http://localhost:7000/my_library/foo?arg1=something,arg2=something >> ( Just wanted a frank discussion on this idea .... ) > > Except that you've also got to transfer the data and the authentication > back and forth. You need much more than sysfs for this, you need some way > to pass messages securely, and to do work on a remote node on your > behalf. You need some way to locate a resource. > > It's not entirely impossible to make syscalls remotable. It's been done to > some extent at least two ways. Read up on plan 9 (which did try and take > Unix much more networked and within the classic tradition of Unix as a > small clean OS), and perhaps also on MOSIX, which took a very different > approach to running on a cluster of boxes as one instance. Another odd > example to look at is Minix. Minix treats all syscalls internally as > messages between user processes and services like the file system. Minix > never really made use of it, but it does mean its much easier to do > remote syscall models in Minix than Linux. (in part I suspect this comes > from Tanenbaum's Amoeba and similar work). > > I'm not sure I'd pick Java for this either - Java is very weak in this > area as it tends to expose notions of locality and what is and isn't > remote when doing method invocations. Languages like smalltalk on the > other hand got this right many years before. > > Alan > On Wednesday Jan 7th Siddhartha De (siddhartha.de87 at gmail.com ) wrote :- I have already asked this question on the kernelnewbies IRC but I wanted a bit of feedback from the Linux gurus ... Is sysfs 'supposed' to be a read only where various kernel parameters are exposed to be read ? Or are they 'supposed' to be writable too ? Are all drivers in future required to expose interfaces in sysfs ? If the answer to the above is true , then would we be really requiring IOCTL calls at all anymore ... because all that we would need to do to make a driver do something is to change values in some of the files in sysfs ... for eg to call a ioctl called DO_SOMETHING on driver D1 we would be doing : echo 1> /sysfs/D1/IOCTLS/DO_SOMETHING ( or something like that ... :) ) Are there any plans of exposing the kernel api ( that is syscalls and libc ) as sysfs files ... for eg echo 1> /sysfs/libc/get_system_time cat /sysfs/libc/results/system_time Would it be a good idea to do this ? How about exposing core system calls , libc and any new libraries over http/over the network ? This would make calling ioctls and really low level stuff from high level languages like Java a trivial process . For eg right now if I wanted to call some function say foo () in library my_library.so in Java I would have to take the JNI/JNA route ... doable but not exactly 'easy' ( same for other high level languages ) ... whereas in the new way all I would have to do is send a request to say :- http://localhost:7000/my_library/foo?arg1=something,arg2=something ( Just wanted a frank discussion on this idea .... ) Thanking you in anticipation , Yours sincerely , Kernel newbie ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Fwd: Question about kernel interfaces 2015-01-09 12:30 ` Siddhartha De @ 2015-01-09 14:03 ` Valdis.Kletnieks at vt.edu -1 siblings, 0 replies; 18+ messages in thread From: Valdis.Kletnieks @ 2015-01-09 14:03 UTC (permalink / raw) To: Siddhartha De; +Cc: linux-kernel, kernelnewbies [-- Attachment #1: Type: text/plain, Size: 751 bytes --] On Fri, 09 Jan 2015 18:00:07 +0530, Siddhartha De said: > In what timeframe should we migrate to "pure" sysfs and drop using > any ioctls (if a sysfs interface exists)? Pretty much the answer is "you don't", for the wsame reason that there's still a lot of non-process stuff in /proc - there's no good way to clean up /pron up / proc or get rid of ioctl()'s without breaking userspace. And some ioctl's you can't get rid of because they're part of official standards (for instance, all to TIO_FOO stuff for controlling a terminal are part of POSIX - removing them would break code that used to be portable... Or to put it differently - ioctl() virst appeared in Unix Version 7 in 1979. >From that, calculate how long it will take to get rid of. [-- Attachment #2: Type: application/pgp-signature, Size: 848 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Fwd: Question about kernel interfaces @ 2015-01-09 14:03 ` Valdis.Kletnieks at vt.edu 0 siblings, 0 replies; 18+ messages in thread From: Valdis.Kletnieks at vt.edu @ 2015-01-09 14:03 UTC (permalink / raw) To: kernelnewbies On Fri, 09 Jan 2015 18:00:07 +0530, Siddhartha De said: > In what timeframe should we migrate to "pure" sysfs and drop using > any ioctls (if a sysfs interface exists)? Pretty much the answer is "you don't", for the wsame reason that there's still a lot of non-process stuff in /proc - there's no good way to clean up /pron up / proc or get rid of ioctl()'s without breaking userspace. And some ioctl's you can't get rid of because they're part of official standards (for instance, all to TIO_FOO stuff for controlling a terminal are part of POSIX - removing them would break code that used to be portable... Or to put it differently - ioctl() virst appeared in Unix Version 7 in 1979. >From that, calculate how long it will take to get rid of. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 848 bytes Desc: not available Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20150109/d28fabcd/attachment.bin ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Question about kernel interfaces 2015-01-09 11:48 ` Siddhartha De 2015-01-09 12:20 ` Siddhartha De @ 2015-01-14 16:33 ` One Thousand Gnomes 2015-01-16 19:37 ` Fwd: " Siddhartha De 1 sibling, 1 reply; 18+ messages in thread From: One Thousand Gnomes @ 2015-01-14 16:33 UTC (permalink / raw) To: Siddhartha De; +Cc: linux-kernel > Just wanted to point out something :- > > Quoting from makelinux website :- > http://www.makelinux.net/books/lkd2/ch17lev1sec8 > > "The sysfs filesystem is currently the place for implementing > functionality previously reserved for ioctl() calls on device nodes or > the procfs filesystem. These days, the chic thing to do is implement > such functionality as sysfs attributes in the appropriate directory. > For example, instead of a new ioctl() on a device node, add a sysfs > attribute in the driver's sysfs directory. Such an approach avoids the > type-unsafe use of obscure ioctl() arguments and the haphazard mess of > /proc. " And is wrong. Certainly rather clueless people had the idea they could make ioctl go away using sysfs, and then everyone wrote about it. Anyone who actually thought about the problem realised immediately they are not semantically equivalent. > sysfs is the new ioctl And as was pointed out even then they are not semantically equivalent so everyone who was talking about "sysfs being the new ioctl" was basically talking out the wrong end. What was done was to stop filling drivers with ioctls for pieces of general information which didn't have a context attached to it or sequence. Things like power state, what type it is, what bus it is on etc. That avoided having zillions of ioctls but at the cost of memory. > Taking this idea further , could we also expose driver IOCTL's over > the network ? > > For eg to open the cd rom drive we would be calling the following url :- You could but why would you make the network visible. If you look at real clustered systems they all hide the network. You don't care where it is, often you don't even care where the drive is. eject /dev/cdrom on such a system lets the kernel figure out who to ask to eject it. The job of an OS is to abstract details like that. In a clustered OS I want to be able to type mount mydisk and I don't care who, where or how it gets mounted so long as it appears. Alan ^ permalink raw reply [flat|nested] 18+ messages in thread
* Fwd: Question about kernel interfaces 2015-01-14 16:33 ` One Thousand Gnomes @ 2015-01-16 19:37 ` Siddhartha De 2015-01-18 9:33 ` Siddhartha De 0 siblings, 1 reply; 18+ messages in thread From: Siddhartha De @ 2015-01-16 19:37 UTC (permalink / raw) To: kernelnewbies FYI ... ---------- Forwarded message ---------- From: One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> Date: Wed, Jan 14, 2015 at 10:03 PM Subject: Re: Question about kernel interfaces To: Siddhartha De <siddhartha.de87@gmail.com> Cc: linux-kernel <linux-kernel@vger.kernel.org> > Just wanted to point out something :- > > Quoting from makelinux website :- > http://www.makelinux.net/books/lkd2/ch17lev1sec8 > > "The sysfs filesystem is currently the place for implementing > functionality previously reserved for ioctl() calls on device nodes or > the procfs filesystem. These days, the chic thing to do is implement > such functionality as sysfs attributes in the appropriate directory. > For example, instead of a new ioctl() on a device node, add a sysfs > attribute in the driver's sysfs directory. Such an approach avoids the > type-unsafe use of obscure ioctl() arguments and the haphazard mess of > /proc. " And is wrong. Certainly rather clueless people had the idea they could make ioctl go away using sysfs, and then everyone wrote about it. Anyone who actually thought about the problem realised immediately they are not semantically equivalent. > sysfs is the new ioctl And as was pointed out even then they are not semantically equivalent so everyone who was talking about "sysfs being the new ioctl" was basically talking out the wrong end. What was done was to stop filling drivers with ioctls for pieces of general information which didn't have a context attached to it or sequence. Things like power state, what type it is, what bus it is on etc. That avoided having zillions of ioctls but at the cost of memory. > Taking this idea further , could we also expose driver IOCTL's over > the network ? > > For eg to open the cd rom drive we would be calling the following url :- You could but why would you make the network visible. If you look at real clustered systems they all hide the network. You don't care where it is, often you don't even care where the drive is. eject /dev/cdrom on such a system lets the kernel figure out who to ask to eject it. The job of an OS is to abstract details like that. In a clustered OS I want to be able to type mount mydisk and I don't care who, where or how it gets mounted so long as it appears. Alan ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Question about kernel interfaces 2015-01-16 19:37 ` Fwd: " Siddhartha De @ 2015-01-18 9:33 ` Siddhartha De 0 siblings, 0 replies; 18+ messages in thread From: Siddhartha De @ 2015-01-18 9:33 UTC (permalink / raw) To: kernelnewbies, linux-kernel, One Thousand Gnomes Hi Alan Thanks for your reply ... really helped a lot >> Taking this idea further , could we also expose driver IOCTL's over >> the network ? >> >> For eg to open the cd rom drive we would be calling the following url :- > > You could but why would you make the network visible. If you look at real > clustered systems they all hide the network. You don't care where it is, > often you don't even care where the drive is. > > eject /dev/cdrom > > on such a system lets the kernel figure out who to ask to eject it. > > The job of an OS is to abstract details like that. In a clustered OS I > want to be able to type > > mount mydisk > > and I don't care who, where or how it gets mounted so long as it appears. > > Alan OK ... the idea of exposing driver ioctl 's or low level system calls or library calls over the network isn't for use in clustered systems ( although it certainly could be used there ) ... the reason I want to expose all these systems is to provide an universal interface that any programming language can request services from . To bring home the point to the Linux folks let's talk about a particular example .... Let's say you need to call an ioctl from a shell script ( I know its a very rare use case but please bear with me ... :) ) So the current way of doing it is probably to write a C program which actually calls the ioctl and then call the C program from the shell script ... However if we exposed the IOCTL as a web url we could easily call its services by using a simple utility like curl . So in our shell script we would probably write :- curl http://localhost:7000/IOCTL/IOCTL_EJECT?device=/dev/cdrom and the IOCTL_EJECT ioctl would get called on /dev/cdrom that is ioctl ( &handle_to_/dev/cdrom , IOCTL_EJECT , &some_buffer ) would get called ( by our translator running as a separate program or as a service daemon ) Taking this concept further all libraries could expose their functions via such an interface . And the benefits would go far beyond just shell scripts . Once the interface has been exposed any high level language can use the functionality provided by drivers , libraries to their full extent . If we want to expose the full power of drivers and other libraries to the entire gamut of programmers regardless of whatever language they are coding in , this could be a good way of doing it :) Thanking you , Yours sincerely , Kernel Newbie On Sat, Jan 17, 2015 at 1:07 AM, Siddhartha De <siddhartha.de87@gmail.com> wrote: > FYI ... > > > ---------- Forwarded message ---------- > From: One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> > Date: Wed, Jan 14, 2015 at 10:03 PM > Subject: Re: Question about kernel interfaces > To: Siddhartha De <siddhartha.de87@gmail.com> > Cc: linux-kernel <linux-kernel@vger.kernel.org> > > >> Just wanted to point out something :- >> >> Quoting from makelinux website :- >> http://www.makelinux.net/books/lkd2/ch17lev1sec8 >> >> "The sysfs filesystem is currently the place for implementing >> functionality previously reserved for ioctl() calls on device nodes or >> the procfs filesystem. These days, the chic thing to do is implement >> such functionality as sysfs attributes in the appropriate directory. >> For example, instead of a new ioctl() on a device node, add a sysfs >> attribute in the driver's sysfs directory. Such an approach avoids the >> type-unsafe use of obscure ioctl() arguments and the haphazard mess of >> /proc. " > > And is wrong. Certainly rather clueless people had the idea they could > make ioctl go away using sysfs, and then everyone wrote about it. Anyone > who actually thought about the problem realised immediately they are not > semantically equivalent. > >> sysfs is the new ioctl > > And as was pointed out even then they are not semantically equivalent so > everyone who was talking about "sysfs being the new ioctl" was basically > talking out the wrong end. > > What was done was to stop filling drivers with ioctls for pieces of > general information which didn't have a context attached to it or > sequence. Things like power state, what type it is, what bus it is on > etc. That avoided having zillions of ioctls but at the cost of memory. > >> Taking this idea further , could we also expose driver IOCTL's over >> the network ? >> >> For eg to open the cd rom drive we would be calling the following url :- > > You could but why would you make the network visible. If you look at real > clustered systems they all hide the network. You don't care where it is, > often you don't even care where the drive is. > > eject /dev/cdrom > > on such a system lets the kernel figure out who to ask to eject it. > > The job of an OS is to abstract details like that. In a clustered OS I > want to be able to type > > mount mydisk > > and I don't care who, where or how it gets mounted so long as it appears. > > Alan ^ permalink raw reply [flat|nested] 18+ messages in thread
* Question about kernel interfaces @ 2015-01-18 9:33 ` Siddhartha De 0 siblings, 0 replies; 18+ messages in thread From: Siddhartha De @ 2015-01-18 9:33 UTC (permalink / raw) To: kernelnewbies Hi Alan Thanks for your reply ... really helped a lot >> Taking this idea further , could we also expose driver IOCTL's over >> the network ? >> >> For eg to open the cd rom drive we would be calling the following url :- > > You could but why would you make the network visible. If you look at real > clustered systems they all hide the network. You don't care where it is, > often you don't even care where the drive is. > > eject /dev/cdrom > > on such a system lets the kernel figure out who to ask to eject it. > > The job of an OS is to abstract details like that. In a clustered OS I > want to be able to type > > mount mydisk > > and I don't care who, where or how it gets mounted so long as it appears. > > Alan OK ... the idea of exposing driver ioctl 's or low level system calls or library calls over the network isn't for use in clustered systems ( although it certainly could be used there ) ... the reason I want to expose all these systems is to provide an universal interface that any programming language can request services from . To bring home the point to the Linux folks let's talk about a particular example .... Let's say you need to call an ioctl from a shell script ( I know its a very rare use case but please bear with me ... :) ) So the current way of doing it is probably to write a C program which actually calls the ioctl and then call the C program from the shell script ... However if we exposed the IOCTL as a web url we could easily call its services by using a simple utility like curl . So in our shell script we would probably write :- curl http://localhost:7000/IOCTL/IOCTL_EJECT?device=/dev/cdrom and the IOCTL_EJECT ioctl would get called on /dev/cdrom that is ioctl ( &handle_to_/dev/cdrom , IOCTL_EJECT , &some_buffer ) would get called ( by our translator running as a separate program or as a service daemon ) Taking this concept further all libraries could expose their functions via such an interface . And the benefits would go far beyond just shell scripts . Once the interface has been exposed any high level language can use the functionality provided by drivers , libraries to their full extent . If we want to expose the full power of drivers and other libraries to the entire gamut of programmers regardless of whatever language they are coding in , this could be a good way of doing it :) Thanking you , Yours sincerely , Kernel Newbie On Sat, Jan 17, 2015 at 1:07 AM, Siddhartha De <siddhartha.de87@gmail.com> wrote: > FYI ... > > > ---------- Forwarded message ---------- > From: One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> > Date: Wed, Jan 14, 2015 at 10:03 PM > Subject: Re: Question about kernel interfaces > To: Siddhartha De <siddhartha.de87@gmail.com> > Cc: linux-kernel <linux-kernel@vger.kernel.org> > > >> Just wanted to point out something :- >> >> Quoting from makelinux website :- >> http://www.makelinux.net/books/lkd2/ch17lev1sec8 >> >> "The sysfs filesystem is currently the place for implementing >> functionality previously reserved for ioctl() calls on device nodes or >> the procfs filesystem. These days, the chic thing to do is implement >> such functionality as sysfs attributes in the appropriate directory. >> For example, instead of a new ioctl() on a device node, add a sysfs >> attribute in the driver's sysfs directory. Such an approach avoids the >> type-unsafe use of obscure ioctl() arguments and the haphazard mess of >> /proc. " > > And is wrong. Certainly rather clueless people had the idea they could > make ioctl go away using sysfs, and then everyone wrote about it. Anyone > who actually thought about the problem realised immediately they are not > semantically equivalent. > >> sysfs is the new ioctl > > And as was pointed out even then they are not semantically equivalent so > everyone who was talking about "sysfs being the new ioctl" was basically > talking out the wrong end. > > What was done was to stop filling drivers with ioctls for pieces of > general information which didn't have a context attached to it or > sequence. Things like power state, what type it is, what bus it is on > etc. That avoided having zillions of ioctls but at the cost of memory. > >> Taking this idea further , could we also expose driver IOCTL's over >> the network ? >> >> For eg to open the cd rom drive we would be calling the following url :- > > You could but why would you make the network visible. If you look at real > clustered systems they all hide the network. You don't care where it is, > often you don't even care where the drive is. > > eject /dev/cdrom > > on such a system lets the kernel figure out who to ask to eject it. > > The job of an OS is to abstract details like that. In a clustered OS I > want to be able to type > > mount mydisk > > and I don't care who, where or how it gets mounted so long as it appears. > > Alan ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Question about kernel interfaces 2015-01-18 9:33 ` Siddhartha De @ 2015-01-18 22:03 ` Valdis.Kletnieks at vt.edu -1 siblings, 0 replies; 18+ messages in thread From: Valdis.Kletnieks @ 2015-01-18 22:03 UTC (permalink / raw) To: Siddhartha De; +Cc: kernelnewbies, linux-kernel, One Thousand Gnomes [-- Attachment #1: Type: text/plain, Size: 1483 bytes --] On Sun, 18 Jan 2015 15:03:55 +0530, Siddhartha De said: > Let's say you need to call an ioctl from a shell script ( I know its a > very rare use case but please bear with me ... :) ) There's a *reason* it's very rare... > So the current way of doing it is probably to write a C program which > actually calls the ioctl and then call the C program from the shell > script ... > > However if we exposed the IOCTL as a web url we could easily call its > services by using a simple utility like curl . So in our shell script > we would probably write :- > > curl http://localhost:7000/IOCTL/IOCTL_EJECT?device=/dev/cdrom > > and the IOCTL_EJECT ioctl would get called on /dev/cdrom > that is ioctl ( &handle_to_/dev/cdrom , IOCTL_EJECT , &some_buffer ) > would get called ( by our translator running as a separate program or > as a service daemon ) > Why use curl when /usr/bin/eject is almost certainly available? Can you point at an *actual* case where (a) shell script access even makes *sense*, and also (b) there *isn't* already a program available for the shell script to call to issue the ioctl? Or point at a non-shell language that *doesn't* have already have access to the ioctl() wrapper that's in whatever libc you have? I think you're trying to solve a non-problem here. (For bonus points, instead of EJECT, explain how you form a curl URL that does TCSETS. Yes, you *do* have to handle *all* the fields of a struct termios, for both 32 and 64 bit environments....) [-- Attachment #2: Type: application/pgp-signature, Size: 848 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Question about kernel interfaces @ 2015-01-18 22:03 ` Valdis.Kletnieks at vt.edu 0 siblings, 0 replies; 18+ messages in thread From: Valdis.Kletnieks at vt.edu @ 2015-01-18 22:03 UTC (permalink / raw) To: kernelnewbies On Sun, 18 Jan 2015 15:03:55 +0530, Siddhartha De said: > Let's say you need to call an ioctl from a shell script ( I know its a > very rare use case but please bear with me ... :) ) There's a *reason* it's very rare... > So the current way of doing it is probably to write a C program which > actually calls the ioctl and then call the C program from the shell > script ... > > However if we exposed the IOCTL as a web url we could easily call its > services by using a simple utility like curl . So in our shell script > we would probably write :- > > curl http://localhost:7000/IOCTL/IOCTL_EJECT?device=/dev/cdrom > > and the IOCTL_EJECT ioctl would get called on /dev/cdrom > that is ioctl ( &handle_to_/dev/cdrom , IOCTL_EJECT , &some_buffer ) > would get called ( by our translator running as a separate program or > as a service daemon ) > Why use curl when /usr/bin/eject is almost certainly available? Can you point at an *actual* case where (a) shell script access even makes *sense*, and also (b) there *isn't* already a program available for the shell script to call to issue the ioctl? Or point at a non-shell language that *doesn't* have already have access to the ioctl() wrapper that's in whatever libc you have? I think you're trying to solve a non-problem here. (For bonus points, instead of EJECT, explain how you form a curl URL that does TCSETS. Yes, you *do* have to handle *all* the fields of a struct termios, for both 32 and 64 bit environments....) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 848 bytes Desc: not available Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20150118/e132f3a8/attachment.bin ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Question about kernel interfaces 2015-01-18 9:33 ` Siddhartha De (?) (?) @ 2015-01-21 17:44 ` One Thousand Gnomes 2015-02-07 15:20 ` Siddhartha De -1 siblings, 1 reply; 18+ messages in thread From: One Thousand Gnomes @ 2015-01-21 17:44 UTC (permalink / raw) To: Siddhartha De; +Cc: kernelnewbies, linux-kernel On Sun, 18 Jan 2015 15:03:55 +0530 > Let's say you need to call an ioctl from a shell script ( I know its a > very rare use case but please bear with me ... :) ) > > So the current way of doing it is probably to write a C program which > actually calls the ioctl and then call the C program from the shell > script ... That's how Unix is designed yes. It means that the ioctl internals are hidden from the scripts and the tools provided can be used instead. > and the IOCTL_EJECT ioctl would get called on /dev/cdrom > that is ioctl ( &handle_to_/dev/cdrom , IOCTL_EJECT , &some_buffer ) > would get called ( by our translator running as a separate program or > as a service daemon ) You can just run "eject". It knows about all the details of doing an eject. The equivalent of a "library" in shell scripts is a program. Small, simple tools that do one job and do it right, or at least before GNU got hold of them 8) Alan ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Question about kernel interfaces 2015-01-21 17:44 ` One Thousand Gnomes @ 2015-02-07 15:20 ` Siddhartha De 0 siblings, 0 replies; 18+ messages in thread From: Siddhartha De @ 2015-02-07 15:20 UTC (permalink / raw) To: One Thousand Gnomes, Valdis.Kletnieks@vt.edu; +Cc: kernelnewbies, linux-kernel >(For bonus points, instead of EJECT, explain how you form a curl URL >that does TCSETS. Yes, you *do* have to handle *all* the fields of a >struct termios, for both 32 and 64 bit environments....) I am new to termios programming so please correct me if I am wrong ... I did a bit of googling and well here's a probable curl url and the corresponding pseudocode that needs to be in our interpreter curl http://localhost:7000/IOCTL/TCSETS2?file_descriptor=fd&cflag=something&ispeed=something&ospeed=something And here's a little pseudocode for our interpreter :- struct termios2 tio; // ioctl(fd, TCGETS2, &tio); // assuming get_from_url is a function which parses the incomming url and returns the required parameters cflag=get_from_url (cflag ) ; ispeed = get_from_url ( ispeed ) ; ospeed=get_from_url(ospeed) tio.c_cflag = cflag ; tio.c_ispeed = ispeed; tio.c_ospeed = ospeed; ioctl(fd, TCSETS2, &tio); Thanking you in anticipation , Yours sincerely , Kernel newbie On Wed, Jan 21, 2015 at 11:14 PM, One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote: > On Sun, 18 Jan 2015 15:03:55 +0530 >> Let's say you need to call an ioctl from a shell script ( I know its a >> very rare use case but please bear with me ... :) ) >> >> So the current way of doing it is probably to write a C program which >> actually calls the ioctl and then call the C program from the shell >> script ... > > That's how Unix is designed yes. It means that the ioctl internals are > hidden from the scripts and the tools provided can be used instead. > >> and the IOCTL_EJECT ioctl would get called on /dev/cdrom >> that is ioctl ( &handle_to_/dev/cdrom , IOCTL_EJECT , &some_buffer ) >> would get called ( by our translator running as a separate program or >> as a service daemon ) > > You can just run "eject". It knows about all the details of doing an > eject. > > The equivalent of a "library" in shell scripts is a program. Small, > simple tools that do one job and do it right, or at least before GNU got > hold of them 8) > > Alan ^ permalink raw reply [flat|nested] 18+ messages in thread
* Question about kernel interfaces @ 2015-02-07 15:20 ` Siddhartha De 0 siblings, 0 replies; 18+ messages in thread From: Siddhartha De @ 2015-02-07 15:20 UTC (permalink / raw) To: kernelnewbies >(For bonus points, instead of EJECT, explain how you form a curl URL >that does TCSETS. Yes, you *do* have to handle *all* the fields of a >struct termios, for both 32 and 64 bit environments....) I am new to termios programming so please correct me if I am wrong ... I did a bit of googling and well here's a probable curl url and the corresponding pseudocode that needs to be in our interpreter curl http://localhost:7000/IOCTL/TCSETS2?file_descriptor=fd&cflag=something&ispeed=something&ospeed=something And here's a little pseudocode for our interpreter :- struct termios2 tio; // ioctl(fd, TCGETS2, &tio); // assuming get_from_url is a function which parses the incomming url and returns the required parameters cflag=get_from_url (cflag ) ; ispeed = get_from_url ( ispeed ) ; ospeed=get_from_url(ospeed) tio.c_cflag = cflag ; tio.c_ispeed = ispeed; tio.c_ospeed = ospeed; ioctl(fd, TCSETS2, &tio); Thanking you in anticipation , Yours sincerely , Kernel newbie On Wed, Jan 21, 2015 at 11:14 PM, One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> wrote: > On Sun, 18 Jan 2015 15:03:55 +0530 >> Let's say you need to call an ioctl from a shell script ( I know its a >> very rare use case but please bear with me ... :) ) >> >> So the current way of doing it is probably to write a C program which >> actually calls the ioctl and then call the C program from the shell >> script ... > > That's how Unix is designed yes. It means that the ioctl internals are > hidden from the scripts and the tools provided can be used instead. > >> and the IOCTL_EJECT ioctl would get called on /dev/cdrom >> that is ioctl ( &handle_to_/dev/cdrom , IOCTL_EJECT , &some_buffer ) >> would get called ( by our translator running as a separate program or >> as a service daemon ) > > You can just run "eject". It knows about all the details of doing an > eject. > > The equivalent of a "library" in shell scripts is a program. Small, > simple tools that do one job and do it right, or at least before GNU got > hold of them 8) > > Alan ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2015-02-07 15:20 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <CADxyqMFnDnFNvTL7Yuzd-1WP9RRddFYAd7UVXmLT2SOMRmOx9g@mail.gmail.com> 2015-01-07 12:16 ` Fwd: Question about kernel interfaces Siddhartha De 2015-01-08 13:48 ` One Thousand Gnomes 2015-01-09 11:48 ` Siddhartha De 2015-01-09 12:20 ` Fwd: " Siddhartha De 2015-01-09 12:20 ` Siddhartha De 2015-01-09 12:30 ` Siddhartha De 2015-01-09 12:30 ` Siddhartha De 2015-01-09 14:03 ` Valdis.Kletnieks 2015-01-09 14:03 ` Valdis.Kletnieks at vt.edu 2015-01-14 16:33 ` One Thousand Gnomes 2015-01-16 19:37 ` Fwd: " Siddhartha De 2015-01-18 9:33 ` Siddhartha De 2015-01-18 9:33 ` Siddhartha De 2015-01-18 22:03 ` Valdis.Kletnieks 2015-01-18 22:03 ` Valdis.Kletnieks at vt.edu 2015-01-21 17:44 ` One Thousand Gnomes 2015-02-07 15:20 ` Siddhartha De 2015-02-07 15:20 ` Siddhartha De
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.