* creating live virtual files by concatenation @ 2006-02-25 14:37 Maciej Soltysiak 2006-02-25 15:30 ` Jan Engelhardt ` (2 more replies) 0 siblings, 3 replies; 15+ messages in thread From: Maciej Soltysiak @ 2006-02-25 14:37 UTC (permalink / raw) To: linux-kernel; +Cc: reiserfs-list Hello! I have this idea about creating sort of a virtual file. Let us say I have three text files that contain javascript code: tooltip.js banner.js foo.js Now let us say I am creating sort of a virtual text file (code.js) that is a live-concatenation of these files: # concatenate tooltip.js banner.js foo.js code.js Note I am not talking about the cat(1) utility. I am thinking of code.js be always a live concatenated version of these three, so when I modify one file, the live-version is also modified. What puprose I might have? Network-related. Say, I have an HTML file that includes these three files in its code. When a browser downloads the HTML file it will then create three threads to download each of those javascript files. If I had a live-concatenated file, I could reference it in the HTML file so that the browser does not have to download three files but just one. This would surely reduce network overhead of downloading the same amount of data but within just one connection, reduce resource usage on the client and possibly (depending on implementation) reduce the cost of accessing three individual files on the server. I am CC'ing reiserfs-list because Reiser4 would seem to be the most robust filesystem that could have it done. Any thoughts about the idea itself? Would be nice if this idea could inspire some talented hackers here and there. Best Regards, Maciej ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: creating live virtual files by concatenation 2006-02-25 14:37 creating live virtual files by concatenation Maciej Soltysiak @ 2006-02-25 15:30 ` Jan Engelhardt 2006-02-27 19:52 ` Yoanis Gil Delgado 2006-02-25 15:35 ` Jesper Juhl 2006-02-25 17:40 ` Peter Foldiak 2 siblings, 1 reply; 15+ messages in thread From: Jan Engelhardt @ 2006-02-25 15:30 UTC (permalink / raw) To: Maciej Soltysiak; +Cc: linux-kernel, reiserfs-list > >Now let us say I am creating sort of a virtual text file (code.js) >that is a live-concatenation of these files: ># concatenate tooltip.js banner.js foo.js code.js > >Note I am not talking about the cat(1) utility. I am thinking of >code.js be always a live concatenated version of these three, so when >I modify one file, the live-version is also modified. > >What puprose I might have? Network-related. Say, I have an HTML file >that includes these three files in its code. > Try FUSE. >If I had a live-concatenated file, I could reference it in the HTML file >so that the browser does not have to download three files but just one. > >This would surely reduce network overhead of downloading the same amount >of data but within just one connection, reduce resource usage on the client >and possibly (depending on implementation) reduce the cost of accessing >three individual files on the server. > Have you ever heard of persistent connections with HTTP/1.1? Jan Engelhardt -- ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: creating live virtual files by concatenation 2006-02-25 15:30 ` Jan Engelhardt @ 2006-02-27 19:52 ` Yoanis Gil Delgado 0 siblings, 0 replies; 15+ messages in thread From: Yoanis Gil Delgado @ 2006-02-27 19:52 UTC (permalink / raw) To: Jan Engelhardt; +Cc: Maciej Soltysiak, linux-kernel, reiserfs-list Jan Engelhardt wrote: >>Now let us say I am creating sort of a virtual text file (code.js) >>that is a live-concatenation of these files: >># concatenate tooltip.js banner.js foo.js code.js >> >>Note I am not talking about the cat(1) utility. I am thinking of >>code.js be always a live concatenated version of these three, so when >>I modify one file, the live-version is also modified. >> >>What puprose I might have? Network-related. Say, I have an HTML file >>that includes these three files in its code. >> >> >> >Try FUSE. > > Yes that's the best solution. Email me if you have a question about how to accomplish this. Here at our school we have created a fuse filesystem that "glues" files in a single one. > > >>If I had a live-concatenated file, I could reference it in the HTML file >>so that the browser does not have to download three files but just one. >> >>This would surely reduce network overhead of downloading the same amount >>of data but within just one connection, reduce resource usage on the client >>and possibly (depending on implementation) reduce the cost of accessing >>three individual files on the server. >> >> >> >Have you ever heard of persistent connections with HTTP/1.1? > > >Jan Engelhardt > > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: creating live virtual files by concatenation 2006-02-25 14:37 creating live virtual files by concatenation Maciej Soltysiak 2006-02-25 15:30 ` Jan Engelhardt @ 2006-02-25 15:35 ` Jesper Juhl 2006-02-25 17:15 ` Maciej Soltysiak 2006-02-25 17:40 ` Peter Foldiak 2 siblings, 1 reply; 15+ messages in thread From: Jesper Juhl @ 2006-02-25 15:35 UTC (permalink / raw) To: Maciej Soltysiak; +Cc: linux-kernel, reiserfs-list On 2/25/06, Maciej Soltysiak <solt2@dns.toxicfilms.tv> wrote: > Hello! > > I have this idea about creating sort of a virtual file. > > Let us say I have three text files that contain javascript code: > tooltip.js > banner.js > foo.js > > Now let us say I am creating sort of a virtual text file (code.js) > that is a live-concatenation of these files: > # concatenate tooltip.js banner.js foo.js code.js > > Note I am not talking about the cat(1) utility. I am thinking of > code.js be always a live concatenated version of these three, so when > I modify one file, the live-version is also modified. > > What puprose I might have? Network-related. Say, I have an HTML file > that includes these three files in its code. > > When a browser downloads the HTML file it will then create three threads > to download each of those javascript files. > > If I had a live-concatenated file, I could reference it in the HTML file > so that the browser does not have to download three files but just one. > If that's what you want to accomplish, then you can easily get around that in several ways. 1. Simply $ cat tooltip.js banner.js foo.js > code.js then include code.js in your html document and remember to update it when you change one of the 3 files (or create a script that does it. 2. use Apache's mod_include 3. Use PHP, Perl, python or watever your scripting language of choice is - here's an example in PHP : <?php header('Content-type: text/javascript'); readfile('tooltip.js'); readfile('banner.js'); readfile('foo.js'); ?> save that as javascripts.php then put this in your HTML document : <script src="javascripts.php" language="javascript" type="text/javascript"></script> And there are other ways ... > This would surely reduce network overhead of downloading the same amount > of data but within just one connection, reduce resource usage on the client > and possibly (depending on implementation) reduce the cost of accessing > three individual files on the server. > Negligible I'd say. > I am CC'ing reiserfs-list because Reiser4 would seem to be the most > robust filesystem that could have it done. > > Any thoughts about the idea itself? Might be a cute little hack, but I don't think it's a very useful feature really.. > Would be nice if this idea could inspire some talented hackers here and there. > > Best Regards, > Maciej > -- Jesper Juhl <jesper.juhl@gmail.com> Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html Plain text mails only, please http://www.expita.com/nomime.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: creating live virtual files by concatenation 2006-02-25 15:35 ` Jesper Juhl @ 2006-02-25 17:15 ` Maciej Soltysiak 2006-02-25 18:40 ` Rik van Riel 0 siblings, 1 reply; 15+ messages in thread From: Maciej Soltysiak @ 2006-02-25 17:15 UTC (permalink / raw) To: Jesper Juhl, Jan Engelhardt; +Cc: linux-kernel, reiserfs-list Jesper, Jan, thanks for your replies. What I had in mind was not a solution to any specific problem, but a filesystem feature. A feature that could be used anywhere to have a live version of files, a bit of what SQL CREATE VIEW could do for databases. Code files, DNS zones, configuration files, HTML code. We are still dealing with lots of text files today. Sometimes we do tiresome operations on them or have to make up solutions similar to which Jesper had proposed (using cat and updating it, using application level features,etc) > Might be a cute little hack, but I don't think it's a very useful > feature really.. Well, I will give FUSE a shot. However I have a hunch that this type of feature is actually the future of something like filesystem services for userspace. Filesystems providing ways for userspace applications to create temporary files, view, collections based on metadata. But that is completely an other story, I wanted to keep low with ideas. -- Best regards, Maciej ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: creating live virtual files by concatenation 2006-02-25 17:15 ` Maciej Soltysiak @ 2006-02-25 18:40 ` Rik van Riel 2006-02-25 18:52 ` Jesper Juhl 0 siblings, 1 reply; 15+ messages in thread From: Rik van Riel @ 2006-02-25 18:40 UTC (permalink / raw) To: Maciej Soltysiak; +Cc: Jesper Juhl, Jan Engelhardt, linux-kernel, reiserfs-list On Sat, 25 Feb 2006, Maciej Soltysiak wrote: > Code files, DNS zones, configuration files, HTML code. We are still > dealing with lots of text files today. You say it like it's a bad thing, but in truth I suspect people often deal with text files because they're EASY to manipulate through scripts, etc. -- All Rights Reversed ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: creating live virtual files by concatenation 2006-02-25 18:40 ` Rik van Riel @ 2006-02-25 18:52 ` Jesper Juhl 2006-02-25 19:33 ` Re[2]: " Maciej Soltysiak 0 siblings, 1 reply; 15+ messages in thread From: Jesper Juhl @ 2006-02-25 18:52 UTC (permalink / raw) To: Rik van Riel Cc: Maciej Soltysiak, Jan Engelhardt, linux-kernel, reiserfs-list On 2/25/06, Rik van Riel <riel@redhat.com> wrote: > On Sat, 25 Feb 2006, Maciej Soltysiak wrote: > > > Code files, DNS zones, configuration files, HTML code. We are still > > dealing with lots of text files today. > > You say it like it's a bad thing, but in truth I suspect > people often deal with text files because they're EASY > to manipulate through scripts, etc. > I agree 100%, plain text serial files are *easy*. Given the swiss-army knife of tools we have at our disposal (cat, cut, head, tail, sed, awk, split, sort, grep and many more), plain text files are very easy to manipulate - not to mention write apps to manipulate. And that is a very good thing IMHO. I can imagine quite a mess if I open a file that is really a view of several files and then start manipulating text in it across "actual file" boundaries that could blow up easily. -- Jesper Juhl <jesper.juhl@gmail.com> Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html Plain text mails only, please http://www.expita.com/nomime.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re[2]: creating live virtual files by concatenation 2006-02-25 18:52 ` Jesper Juhl @ 2006-02-25 19:33 ` Maciej Soltysiak 2006-02-25 22:30 ` Peter Foldiak 2006-02-27 20:39 ` Hans Reiser 0 siblings, 2 replies; 15+ messages in thread From: Maciej Soltysiak @ 2006-02-25 19:33 UTC (permalink / raw) To: Jesper Juhl; +Cc: Rik van Riel, Jan Engelhardt, linux-kernel, reiserfs-list Hello Jesper Saturday, February 25, 2006, 7:52:06 PM, you wrote: > On 2/25/06, Rik van Riel <riel@redhat.com> wrote: >> On Sat, 25 Feb 2006, Maciej Soltysiak wrote: >> >> > Code files, DNS zones, configuration files, HTML code. We are still >> > dealing with lots of text files today. >> >> You say it like it's a bad thing, but in truth I suspect >> people often deal with text files because they're EASY >> to manipulate through scripts, etc. Well, I did not mean to sound like that. My emphasis should have been on that it sometimes is tiresome. I have no problems with plain text files, I still am a human being, not an binary/XML parser or whatever :-D > I can imagine quite a mess if I open a file that is really a view of > several files and then start manipulating text in it across "actual > file" boundaries that could blow up easily. Well, I meant that file to be read-only. Just a quick concatentated view for reading. -- Best regards, Maciej ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: creating live virtual files by concatenation 2006-02-25 19:33 ` Re[2]: " Maciej Soltysiak @ 2006-02-25 22:30 ` Peter Foldiak 2006-02-25 22:38 ` Rik van Riel 2006-02-27 20:39 ` Hans Reiser 1 sibling, 1 reply; 15+ messages in thread From: Peter Foldiak @ 2006-02-25 22:30 UTC (permalink / raw) To: Maciej Soltysiak Cc: Jesper Juhl, Rik van Riel, Jan Engelhardt, linux-kernel, reiserfs-list, Hans Reiser Maciej Soltysiak wrote: >>I can imagine quite a mess if I open a file that is really a view of >>several files and then start manipulating text in it across "actual >>file" boundaries that could blow up easily. >> >> >Well, I meant that file to be read-only. Just a quick concatentated view >for reading. > > The quick hack might be useful in certain situations. But the really interesting way to do it would be not to distinguish beween "actual" and "non-actual" files. All of them should be equally "actual", it is just that containment (possibly even overlap) would be allowed. The tree structure used by a file system such as Reiser4 would make this very efficient with each "sub-file" corresponding to a key-range. Writing a chapter should change the book that the chapter is part of. That is what would make it really valuable. Of course it would have all sorts of implications (e.g. for metadata for each part) that need to be thought about, but it could be done properly, I think. Peter ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: creating live virtual files by concatenation 2006-02-25 22:30 ` Peter Foldiak @ 2006-02-25 22:38 ` Rik van Riel 2006-02-25 23:23 ` Peter Foldiak 2006-02-27 20:43 ` Hans Reiser 0 siblings, 2 replies; 15+ messages in thread From: Rik van Riel @ 2006-02-25 22:38 UTC (permalink / raw) To: Peter Foldiak Cc: Maciej Soltysiak, Jesper Juhl, Jan Engelhardt, linux-kernel, reiserfs-list, Hans Reiser On Sat, 25 Feb 2006, Peter Foldiak wrote: > "sub-file" corresponding to a key-range. Writing a chapter should change the > book that the chapter is part of. That is what would make it really valuable. > Of course it would have all sorts of implications (e.g. for metadata for each > part) that need to be thought about, but it could be done properly, I think. What happens if you read the first 10kB of a file, and one of the "chapters" behind your read cursor grows? Do you read part of the same data again when you continue reading? Does the read cursor automatically advance? Your idea changes the way userspace expects files to behave... -- All Rights Reversed ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: creating live virtual files by concatenation 2006-02-25 22:38 ` Rik van Riel @ 2006-02-25 23:23 ` Peter Foldiak 2006-02-27 20:43 ` Hans Reiser 1 sibling, 0 replies; 15+ messages in thread From: Peter Foldiak @ 2006-02-25 23:23 UTC (permalink / raw) To: Rik van Riel Cc: Peter Foldiak, Maciej Soltysiak, Jesper Juhl, Jan Engelhardt, linux-kernel, reiserfs-list, Hans Reiser Rik van Riel wrote: >On Sat, 25 Feb 2006, Peter Foldiak wrote: > >>sub-file" corresponding to a key-range. Writing a chapter should change the >>book that the chapter is part of. That is what would make it really valuable. >>Of course it would have all sorts of implications (e.g. for metadata for each >>part) that need to be thought about, but it could be done properly, I think. >> >> > >What happens if you read the first 10kB of a file, >and one of the "chapters" behind your read cursor >grows? > >Do you read part of the same data again when you >continue reading? > >Does the read cursor automatically advance? > > You should probably continue where you left off . >Your idea changes the way userspace expects files >to behave... > > Yes, I know. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: creating live virtual files by concatenation 2006-02-25 22:38 ` Rik van Riel 2006-02-25 23:23 ` Peter Foldiak @ 2006-02-27 20:43 ` Hans Reiser 1 sibling, 0 replies; 15+ messages in thread From: Hans Reiser @ 2006-02-27 20:43 UTC (permalink / raw) To: Rik van Riel Cc: Peter Foldiak, Maciej Soltysiak, Jesper Juhl, Jan Engelhardt, linux-kernel, reiserfs-list Rik van Riel wrote: >On Sat, 25 Feb 2006, Peter Foldiak wrote: > > > >>"sub-file" corresponding to a key-range. Writing a chapter should change the >>book that the chapter is part of. That is what would make it really valuable. >>Of course it would have all sorts of implications (e.g. for metadata for each >>part) that need to be thought about, but it could be done properly, I think. >> >> > >What happens if you read the first 10kB of a file, >and one of the "chapters" behind your read cursor >grows? > > That is why you have magic delimiters, like the colons and newlines in /etc/passwd. You also specify the delimiters in your inheritance syntax. You don't want it to really all be one file, because you want separation of modification times and acls that let users change their own gecos field and etc., but there are times when you want to act on /etc/passwd as one file. >Do you read part of the same data again when you >continue reading? > >Does the read cursor automatically advance? > >Your idea changes the way userspace expects files >to behave... > > > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: creating live virtual files by concatenation 2006-02-25 19:33 ` Re[2]: " Maciej Soltysiak 2006-02-25 22:30 ` Peter Foldiak @ 2006-02-27 20:39 ` Hans Reiser 2006-02-27 20:45 ` Jesper Juhl 1 sibling, 1 reply; 15+ messages in thread From: Hans Reiser @ 2006-02-27 20:39 UTC (permalink / raw) To: Maciej Soltysiak Cc: Jesper Juhl, Rik van Riel, Jan Engelhardt, linux-kernel, reiserfs-list Pretty much any simple basic tool or extension can be lived without. For them to say that you can do without it is to say nothing. There is little one actually needs a computer to do, I remember professors explaining to me that they felt essays written on a typewritter tended to be better written (it was probably due to them being written by humanities students.;-) ) than those produced using troff. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: creating live virtual files by concatenation 2006-02-27 20:39 ` Hans Reiser @ 2006-02-27 20:45 ` Jesper Juhl 0 siblings, 0 replies; 15+ messages in thread From: Jesper Juhl @ 2006-02-27 20:45 UTC (permalink / raw) To: Hans Reiser Cc: Maciej Soltysiak, Rik van Riel, Jan Engelhardt, linux-kernel, reiserfs-list On 2/27/06, Hans Reiser <reiser@namesys.com> wrote: > Pretty much any simple basic tool or extension can be lived without. > For them to say that you can do without it is to say nothing. There is All I said was that I don't *think* it will be of much use, that's not to say that it *won't* be. As a read-only thing I can certainly see some uses of it - Maciej pointed out some nice ideas, sure, but wether or not they are actually useful when you try to do real work needs to be tested. I for one need to see it and play with it to determine if it will actually be useful. Let's see an implementation and see how it works out for real... -- Jesper Juhl <jesper.juhl@gmail.com> Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html Plain text mails only, please http://www.expita.com/nomime.html ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: creating live virtual files by concatenation 2006-02-25 14:37 creating live virtual files by concatenation Maciej Soltysiak 2006-02-25 15:30 ` Jan Engelhardt 2006-02-25 15:35 ` Jesper Juhl @ 2006-02-25 17:40 ` Peter Foldiak 2 siblings, 0 replies; 15+ messages in thread From: Peter Foldiak @ 2006-02-25 17:40 UTC (permalink / raw) To: reiserfs-list, Maciej Soltysiak; +Cc: linux-kernel hi, I think it is a good idea but there has been quite a long discusion of this issue on the reiserfs-list (some of it also on lkml) in the last few years, with the latest posts in May 2005 I think, most of ithem with the "file as directory" subject . The idea was if you have e.g. a "book" directory/file (object) and chapters within it, then the default action of "book" object when read as a file would be to give a concatenation of the chapter objects within it. There should be ways of specifying non-default actions for the objects too. There has also been a lot a resistance here to this idea, as it is quite a radical change to the concept of the conventional unstructured, serial "file". (I don't believe these are good reason not to try the idea at least.) Read those mails first. Peter On Saturday 25 February 2006 14:37, Maciej Soltysiak wrote: > I have this idea about creating sort of a virtual file. ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2006-02-27 20:45 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2006-02-25 14:37 creating live virtual files by concatenation Maciej Soltysiak 2006-02-25 15:30 ` Jan Engelhardt 2006-02-27 19:52 ` Yoanis Gil Delgado 2006-02-25 15:35 ` Jesper Juhl 2006-02-25 17:15 ` Maciej Soltysiak 2006-02-25 18:40 ` Rik van Riel 2006-02-25 18:52 ` Jesper Juhl 2006-02-25 19:33 ` Re[2]: " Maciej Soltysiak 2006-02-25 22:30 ` Peter Foldiak 2006-02-25 22:38 ` Rik van Riel 2006-02-25 23:23 ` Peter Foldiak 2006-02-27 20:43 ` Hans Reiser 2006-02-27 20:39 ` Hans Reiser 2006-02-27 20:45 ` Jesper Juhl 2006-02-25 17:40 ` Peter Foldiak
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).