Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Robert Hancock]

Yaroslav Rastrigin wrote:
> Well, I could find more or less reasonable explanation of this behaviour - different VM policies of two OSes and 
> strangely strong and persistent belief "Free RAM is a wasted RAM" among kernel devs. Free RAM is not a wasted RAM, its a memory waiting to be used ! 
> Whenever it is needed by apps I'm launching or working with. 

There is no different VM policy here, Windows behaves quite similarly. 
It does not leave memory around unused, it uses it for disk cache.

-- 
Robert Hancock      Saskatoon, SK, Canada
To email, remove "nospam" from hancockr@nospamshaw.ca
Home Page: http://www.roberthancock.com/

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Oliver Neukum]

Am Montag, 9. Januar 2006 15:18 schrieb Robert Hancock:
> Yaroslav Rastrigin wrote:
> > Well, I could find more or less reasonable explanation of this behaviour - different VM policies of two OSes and 
> > strangely strong and persistent belief "Free RAM is a wasted RAM" among kernel devs. Free RAM is not a wasted RAM, its a memory waiting to be used ! 
> > Whenever it is needed by apps I'm launching or working with. 
> 
> There is no different VM policy here, Windows behaves quite similarly. 
> It does not leave memory around unused, it uses it for disk cache.

That doesn't mean that the rate of eviction is the same.
Is it possible that read-ahead is not aggressive enough?

	Regards
		Oliver

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Lee Revell]

On Mon, 2006-01-09 at 15:28 +0100, Oliver Neukum wrote:
> Am Montag, 9. Januar 2006 15:18 schrieb Robert Hancock:
> > Yaroslav Rastrigin wrote:
> > > Well, I could find more or less reasonable explanation of this behaviour - different VM policies of two OSes and 
> > > strangely strong and persistent belief "Free RAM is a wasted RAM" among kernel devs. Free RAM is not a wasted RAM, its a memory waiting to be used ! 
> > > Whenever it is needed by apps I'm launching or working with. 
> > 
> > There is no different VM policy here, Windows behaves quite similarly. 
> > It does not leave memory around unused, it uses it for disk cache.
> 
> That doesn't mean that the rate of eviction is the same.
> Is it possible that read-ahead is not aggressive enough?

Enough for what?  What is the exact problem you are trying to solve?

Lee

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Oliver Neukum]

Am Montag, 9. Januar 2006 16:15 schrieb Lee Revell:
> On Mon, 2006-01-09 at 15:28 +0100, Oliver Neukum wrote:
> > Am Montag, 9. Januar 2006 15:18 schrieb Robert Hancock:
> > > Yaroslav Rastrigin wrote:
> > > > Well, I could find more or less reasonable explanation of this behaviour - different VM policies of two OSes and 
> > > > strangely strong and persistent belief "Free RAM is a wasted RAM" among kernel devs. Free RAM is not a wasted RAM, its a memory waiting to be used ! 
> > > > Whenever it is needed by apps I'm launching or working with. 
> > > 
> > > There is no different VM policy here, Windows behaves quite similarly. 
> > > It does not leave memory around unused, it uses it for disk cache.
> > 
> > That doesn't mean that the rate of eviction is the same.
> > Is it possible that read-ahead is not aggressive enough?
> 
> Enough for what?  What is the exact problem you are trying to solve?

Quicker application startup.

	Regards
		Oliver

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Lee Revell]

On Mon, 2006-01-09 at 17:02 +0100, Oliver Neukum wrote:
> Am Montag, 9. Januar 2006 16:15 schrieb Lee Revell:
> > On Mon, 2006-01-09 at 15:28 +0100, Oliver Neukum wrote:
> > > Am Montag, 9. Januar 2006 15:18 schrieb Robert Hancock:
> > > > Yaroslav Rastrigin wrote:
> > > > > Well, I could find more or less reasonable explanation of this behaviour - different VM policies of two OSes and 
> > > > > strangely strong and persistent belief "Free RAM is a wasted RAM" among kernel devs. Free RAM is not a wasted RAM, its a memory waiting to be used ! 
> > > > > Whenever it is needed by apps I'm launching or working with. 
> > > > 
> > > > There is no different VM policy here, Windows behaves quite similarly. 
> > > > It does not leave memory around unused, it uses it for disk cache.
> > > 
> > > That doesn't mean that the rate of eviction is the same.
> > > Is it possible that read-ahead is not aggressive enough?
> > 
> > Enough for what?  What is the exact problem you are trying to solve?
> 
> Quicker application startup.

Why do you look to the kernel first?  The obvious explanation is that
Linux desktop apps are more bloated than their Windows counterparts.

Lee

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Oliver Neukum]

Am Montag, 9. Januar 2006 17:04 schrieb Lee Revell:
> On Mon, 2006-01-09 at 17:02 +0100, Oliver Neukum wrote:
> > Am Montag, 9. Januar 2006 16:15 schrieb Lee Revell:
> > > On Mon, 2006-01-09 at 15:28 +0100, Oliver Neukum wrote:
> > > > Am Montag, 9. Januar 2006 15:18 schrieb Robert Hancock:
> > > > > Yaroslav Rastrigin wrote:
> > > > > > Well, I could find more or less reasonable explanation of this behaviour - different VM policies of two OSes and 
> > > > > > strangely strong and persistent belief "Free RAM is a wasted RAM" among kernel devs. Free RAM is not a wasted RAM, its a memory waiting to be used ! 
> > > > > > Whenever it is needed by apps I'm launching or working with. 
> > > > > 
> > > > > There is no different VM policy here, Windows behaves quite similarly. 
> > > > > It does not leave memory around unused, it uses it for disk cache.
> > > > 
> > > > That doesn't mean that the rate of eviction is the same.
> > > > Is it possible that read-ahead is not aggressive enough?
> > > 
> > > Enough for what?  What is the exact problem you are trying to solve?
> > 
> > Quicker application startup.
> 
> Why do you look to the kernel first?  The obvious explanation is that
> Linux desktop apps are more bloated than their Windows counterparts.

It is the most efficient place. An improvement to the kernel will improve
all starting times.

	Regards
		Oliver

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Lee Revell]

On Mon, 2006-01-09 at 17:14 +0100, Oliver Neukum wrote:
> Am Montag, 9. Januar 2006 17:04 schrieb Lee Revell:
> > On Mon, 2006-01-09 at 17:02 +0100, Oliver Neukum wrote:
> > > Am Montag, 9. Januar 2006 16:15 schrieb Lee Revell:
> > > > On Mon, 2006-01-09 at 15:28 +0100, Oliver Neukum wrote:
> > > > > Am Montag, 9. Januar 2006 15:18 schrieb Robert Hancock:
> > > > > > Yaroslav Rastrigin wrote:
> > > > > > > Well, I could find more or less reasonable explanation of this behaviour - different VM policies of two OSes and 
> > > > > > > strangely strong and persistent belief "Free RAM is a wasted RAM" among kernel devs. Free RAM is not a wasted RAM, its a memory waiting to be used ! 
> > > > > > > Whenever it is needed by apps I'm launching or working with. 
> > > > > > 
> > > > > > There is no different VM policy here, Windows behaves quite similarly. 
> > > > > > It does not leave memory around unused, it uses it for disk cache.
> > > > > 
> > > > > That doesn't mean that the rate of eviction is the same.
> > > > > Is it possible that read-ahead is not aggressive enough?
> > > > 
> > > > Enough for what?  What is the exact problem you are trying to solve?
> > > 
> > > Quicker application startup.
> > 
> > Why do you look to the kernel first?  The obvious explanation is that
> > Linux desktop apps are more bloated than their Windows counterparts.
> 
> It is the most efficient place. An improvement to the kernel will improve
> all starting times.

I think you'll get at most a 10% or 20% speedup by improving the kernel,
while some of these apps (think Nautilus vs Windows Explorer) will need
to be 1000% faster to seem reasonable to a Windows user.

Lee


Lee

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Bernd Petrovitsch]

On Mon, 2006-01-09 at 11:19 -0500, Lee Revell wrote:
> On Mon, 2006-01-09 at 17:14 +0100, Oliver Neukum wrote:
> > Am Montag, 9. Januar 2006 17:04 schrieb Lee Revell:
> > > On Mon, 2006-01-09 at 17:02 +0100, Oliver Neukum wrote:
> > > > Am Montag, 9. Januar 2006 16:15 schrieb Lee Revell:
> > > > > On Mon, 2006-01-09 at 15:28 +0100, Oliver Neukum wrote:
> > > > > > Am Montag, 9. Januar 2006 15:18 schrieb Robert Hancock:
> > > > > > > Yaroslav Rastrigin wrote:
> > > > > > > > Well, I could find more or less reasonable explanation of this behaviour - different VM policies of two OSes and 
> > > > > > > > strangely strong and persistent belief "Free RAM is a wasted RAM" among kernel devs. Free RAM is not a wasted RAM, its a memory waiting to be used ! 
> > > > > > > > Whenever it is needed by apps I'm launching or working with. 
> > > > > > > 
> > > > > > > There is no different VM policy here, Windows behaves quite similarly. 
> > > > > > > It does not leave memory around unused, it uses it for disk cache.
> > > > > > 
> > > > > > That doesn't mean that the rate of eviction is the same.
> > > > > > Is it possible that read-ahead is not aggressive enough?
> > > > > 
> > > > > Enough for what?  What is the exact problem you are trying to solve?
> > > > 
> > > > Quicker application startup.
> > > 
> > > Why do you look to the kernel first?  The obvious explanation is that
> > > Linux desktop apps are more bloated than their Windows counterparts.
> > 
> > It is the most efficient place. An improvement to the kernel will improve
> > all starting times.
> 
> I think you'll get at most a 10% or 20% speedup by improving the kernel,
> while some of these apps (think Nautilus vs Windows Explorer) will need
> to be 1000% faster to seem reasonable to a Windows user.

That's easy: Just start nautilus, OOorg, Firefox, a java-vm and
GNOME/KDE infrastructure at login time in the background (*eg* and
mlockall() the more important ones so that the are surely in RAM) and
"starting the app" is only a small program connecting to the respective
process to get a fork() there (e.g. like the "-remote" parameter in the
Mozilla family).

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Lee Revell]

On Mon, 2006-01-09 at 17:29 +0100, Bernd Petrovitsch wrote:
> On Mon, 2006-01-09 at 11:19 -0500, Lee Revell wrote:
> > On Mon, 2006-01-09 at 17:14 +0100, Oliver Neukum wrote:
> > > Am Montag, 9. Januar 2006 17:04 schrieb Lee Revell:
> > > > On Mon, 2006-01-09 at 17:02 +0100, Oliver Neukum wrote:
> > > > > Am Montag, 9. Januar 2006 16:15 schrieb Lee Revell:
> > > > > > On Mon, 2006-01-09 at 15:28 +0100, Oliver Neukum wrote:
> > > > > > > Am Montag, 9. Januar 2006 15:18 schrieb Robert Hancock:
> > > > > > > > Yaroslav Rastrigin wrote:
> > > > > > > > > Well, I could find more or less reasonable explanation of this behaviour - different VM policies of two OSes and 
> > > > > > > > > strangely strong and persistent belief "Free RAM is a wasted RAM" among kernel devs. Free RAM is not a wasted RAM, its a memory waiting to be used ! 
> > > > > > > > > Whenever it is needed by apps I'm launching or working with. 
> > > > > > > > 
> > > > > > > > There is no different VM policy here, Windows behaves quite similarly. 
> > > > > > > > It does not leave memory around unused, it uses it for disk cache.
> > > > > > > 
> > > > > > > That doesn't mean that the rate of eviction is the same.
> > > > > > > Is it possible that read-ahead is not aggressive enough?
> > > > > > 
> > > > > > Enough for what?  What is the exact problem you are trying to solve?
> > > > > 
> > > > > Quicker application startup.
> > > > 
> > > > Why do you look to the kernel first?  The obvious explanation is that
> > > > Linux desktop apps are more bloated than their Windows counterparts.
> > > 
> > > It is the most efficient place. An improvement to the kernel will improve
> > > all starting times.
> > 
> > I think you'll get at most a 10% or 20% speedup by improving the kernel,
> > while some of these apps (think Nautilus vs Windows Explorer) will need
> > to be 1000% faster to seem reasonable to a Windows user.
> 
> That's easy: Just start nautilus, OOorg, Firefox, a java-vm and
> GNOME/KDE infrastructure at login time in the background (*eg* and
> mlockall() the more important ones so that the are surely in RAM) and
> "starting the app" is only a small program connecting to the respective
> process to get a fork() there (e.g. like the "-remote" parameter in the
> Mozilla family).

Have you tried this?  I suspect it still takes at least twice as long as
on windows.

For example on my system there was already a "nautilus" process but
"Places -> Home Folder" still took ~2 seconds to display anything, and
~8 seconds to completely render the window and icons.  On Windows this
takes much less than a second.

Lee

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Jeff V. Merkey]

Alan Cox wrote:

>On Llu, 2006-01-09 at 17:53 +0100, Oliver Neukum wrote:
>  
>
>>Does the Windows Explorer draw icons based only on name and metadata?
>>    
>>
>
>Sort of. It also plays tricks on the human by working out what icons are
>visible and loading those first then filling in while the user thinks it
>is ready
>
>  
>

And the mouse driver is biased to always get access to CPU cycles so the 
cursor will always be visible
and working even when the system is totally locked up.  NTFS performance 
is also totally abysimal
when volumes reach 2TB sizes due to fragmentation of the NTFS 
archtiecture.  A problem not shared
with EXT3 FS's.

J

>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>  
>

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Oliver Neukum]

Am Montag, 9. Januar 2006 17:41 schrieb Lee Revell:
> On Mon, 2006-01-09 at 17:29 +0100, Bernd Petrovitsch wrote:
> > On Mon, 2006-01-09 at 11:19 -0500, Lee Revell wrote:
> > > On Mon, 2006-01-09 at 17:14 +0100, Oliver Neukum wrote:
> > > > Am Montag, 9. Januar 2006 17:04 schrieb Lee Revell:
> > > > > On Mon, 2006-01-09 at 17:02 +0100, Oliver Neukum wrote:
> > > > > > Am Montag, 9. Januar 2006 16:15 schrieb Lee Revell:
> > > > > > > On Mon, 2006-01-09 at 15:28 +0100, Oliver Neukum wrote:
> > > > > > > > Am Montag, 9. Januar 2006 15:18 schrieb Robert Hancock:
> > > > > > > > > Yaroslav Rastrigin wrote:
> > > > > > > > > > Well, I could find more or less reasonable explanation of this behaviour - different VM policies of two OSes and 
> > > > > > > > > > strangely strong and persistent belief "Free RAM is a wasted RAM" among kernel devs. Free RAM is not a wasted RAM, its a memory waiting to be used ! 
> > > > > > > > > > Whenever it is needed by apps I'm launching or working with. 
> > > > > > > > > 
> > > > > > > > > There is no different VM policy here, Windows behaves quite similarly. 
> > > > > > > > > It does not leave memory around unused, it uses it for disk cache.
> > > > > > > > 
> > > > > > > > That doesn't mean that the rate of eviction is the same.
> > > > > > > > Is it possible that read-ahead is not aggressive enough?
> > > > > > > 
> > > > > > > Enough for what?  What is the exact problem you are trying to solve?
> > > > > > 
> > > > > > Quicker application startup.
> > > > > 
> > > > > Why do you look to the kernel first?  The obvious explanation is that
> > > > > Linux desktop apps are more bloated than their Windows counterparts.
> > > > 
> > > > It is the most efficient place. An improvement to the kernel will improve
> > > > all starting times.
> > > 
> > > I think you'll get at most a 10% or 20% speedup by improving the kernel,
> > > while some of these apps (think Nautilus vs Windows Explorer) will need
> > > to be 1000% faster to seem reasonable to a Windows user.
> > 
> > That's easy: Just start nautilus, OOorg, Firefox, a java-vm and
> > GNOME/KDE infrastructure at login time in the background (*eg* and
> > mlockall() the more important ones so that the are surely in RAM) and
> > "starting the app" is only a small program connecting to the respective
> > process to get a fork() there (e.g. like the "-remote" parameter in the
> > Mozilla family).
> 
> Have you tried this?  I suspect it still takes at least twice as long as
> on windows.
> 
> For example on my system there was already a "nautilus" process but
> "Places -> Home Folder" still took ~2 seconds to display anything, and
> ~8 seconds to completely render the window and icons.  On Windows this
> takes much less than a second.

Does the Windows Explorer draw icons based only on name and metadata?

	Regards
		Oliver

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Lee Revell]

On Mon, 2006-01-09 at 17:53 +0100, Oliver Neukum wrote:
> Am Montag, 9. Januar 2006 17:41 schrieb Lee Revell:
> > On Mon, 2006-01-09 at 17:29 +0100, Bernd Petrovitsch wrote:
> > > On Mon, 2006-01-09 at 11:19 -0500, Lee Revell wrote:
> > > > On Mon, 2006-01-09 at 17:14 +0100, Oliver Neukum wrote:
> > > > > Am Montag, 9. Januar 2006 17:04 schrieb Lee Revell:
> > > > > > On Mon, 2006-01-09 at 17:02 +0100, Oliver Neukum wrote:
> > > > > > > Am Montag, 9. Januar 2006 16:15 schrieb Lee Revell:
> > > > > > > > On Mon, 2006-01-09 at 15:28 +0100, Oliver Neukum wrote:
> > > > > > > > > Am Montag, 9. Januar 2006 15:18 schrieb Robert Hancock:
> > > > > > > > > > Yaroslav Rastrigin wrote:
> > > > > > > > > > > Well, I could find more or less reasonable explanation of this behaviour - different VM policies of two OSes and 
> > > > > > > > > > > strangely strong and persistent belief "Free RAM is a wasted RAM" among kernel devs. Free RAM is not a wasted RAM, its a memory waiting to be used ! 
> > > > > > > > > > > Whenever it is needed by apps I'm launching or working with. 
> > > > > > > > > > 
> > > > > > > > > > There is no different VM policy here, Windows behaves quite similarly. 
> > > > > > > > > > It does not leave memory around unused, it uses it for disk cache.
> > > > > > > > > 
> > > > > > > > > That doesn't mean that the rate of eviction is the same.
> > > > > > > > > Is it possible that read-ahead is not aggressive enough?
> > > > > > > > 
> > > > > > > > Enough for what?  What is the exact problem you are trying to solve?
> > > > > > > 
> > > > > > > Quicker application startup.
> > > > > > 
> > > > > > Why do you look to the kernel first?  The obvious explanation is that
> > > > > > Linux desktop apps are more bloated than their Windows counterparts.
> > > > > 
> > > > > It is the most efficient place. An improvement to the kernel will improve
> > > > > all starting times.
> > > > 
> > > > I think you'll get at most a 10% or 20% speedup by improving the kernel,
> > > > while some of these apps (think Nautilus vs Windows Explorer) will need
> > > > to be 1000% faster to seem reasonable to a Windows user.
> > > 
> > > That's easy: Just start nautilus, OOorg, Firefox, a java-vm and
> > > GNOME/KDE infrastructure at login time in the background (*eg* and
> > > mlockall() the more important ones so that the are surely in RAM) and
> > > "starting the app" is only a small program connecting to the respective
> > > process to get a fork() there (e.g. like the "-remote" parameter in the
> > > Mozilla family).
> > 
> > Have you tried this?  I suspect it still takes at least twice as long as
> > on windows.
> > 
> > For example on my system there was already a "nautilus" process but
> > "Places -> Home Folder" still took ~2 seconds to display anything, and
> > ~8 seconds to completely render the window and icons.  On Windows this
> > takes much less than a second.
> 
> Does the Windows Explorer draw icons based only on name and metadata?

No idea.  All i know is it's fast enough to be usable and Nautilus
isn't.

Lee

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Bernd Petrovitsch]

On Mon, 2006-01-09 at 11:41 -0500, Lee Revell wrote:
> On Mon, 2006-01-09 at 17:29 +0100, Bernd Petrovitsch wrote:
> > On Mon, 2006-01-09 at 11:19 -0500, Lee Revell wrote:
[....]
> > > I think you'll get at most a 10% or 20% speedup by improving the kernel,
> > > while some of these apps (think Nautilus vs Windows Explorer) will need
> > > to be 1000% faster to seem reasonable to a Windows user.
> > 
> > That's easy: Just start nautilus, OOorg, Firefox, a java-vm and
> > GNOME/KDE infrastructure at login time in the background (*eg* and
> > mlockall() the more important ones so that the are surely in RAM) and
> > "starting the app" is only a small program connecting to the respective
> > process to get a fork() there (e.g. like the "-remote" parameter in the
> > Mozilla family).
> 
> Have you tried this?  I suspect it still takes at least twice as long as
> on windows.

No, I don't run all on them at once (only xemacs, firefox and the above
forgotten evolution on XFCE) and if, the hosts I'm usually working on
have probably not enough RAM for mlockall(MCL_CURRRENT).
And AFAIK there are no small starter programs and features - only for
Mozilla and cousins.

> For example on my system there was already a "nautilus" process but
> "Places -> Home Folder" still took ~2 seconds to display anything, and
> ~8 seconds to completely render the window and icons.  On Windows this
> takes much less than a second.

Hmm, what happens on the click:
- The click is transported via X, TCP/IP to the application.
- The app decides what to do and generates anew window and content.
- If swapped out, swap it in.
- Load the home directory and what else is needed for displaying it
  (Icons, etc.) from the disk.
- Load these images into the X server.

Id a completely new progrma is started, you must load it from disk, wait
for the shared linker, let it initialize everything and *then* it is
actually usable (and I hate that default OO.org feature that it puts the
window in foreground after startup  at least three times until the
document is loaded).

If you have a memory hog like firefox with a large page loaded then lots
of RAM are occupied anyways and no one except firefox can do anything
about it (without swapping out and this is also slow). And I have just
to switch tabs to hear my disk working.

Ths point is: You have to analyze *where* the apps spend that much time
(independent of being idle wehile waiting for disk I/O or busy moving MB
of graphic data around). It doesn't make that much sense to improve the
(e.g.) 2% in the kernel by 50% since that would bring next to nothing.

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Olivier Galibert]

On Mon, Jan 09, 2006 at 05:53:35PM +0100, Oliver Neukum wrote:
> Does the Windows Explorer draw icons based only on name and metadata?

>From what I can see it does icons on non-executable entirely based on
the extension and nothing else on the first pass.  Executables are
looked inside for an icon (and there seems to be cache effects at
times, especially visible on the desktop).  Then for images a second
pass generates icons depending on the contents (with, once again, a
cache hidden somewhere).

Not a bad strategy, too.  Doing a file(1) on everything can only be
slow given the random disk accesses it generates.  Maybe a file(1) as
a _second_ pass would work.

  OG.

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Bernd Petrovitsch]

On Mon, 2006-01-09 at 17:53 +0100, Oliver Neukum wrote:
> Am Montag, 9. Januar 2006 17:41 schrieb Lee Revell:
> > On Mon, 2006-01-09 at 17:29 +0100, Bernd Petrovitsch wrote:
> > > On Mon, 2006-01-09 at 11:19 -0500, Lee Revell wrote:
> > > > On Mon, 2006-01-09 at 17:14 +0100, Oliver Neukum wrote:
> > > > > Am Montag, 9. Januar 2006 17:04 schrieb Lee Revell:
> > > > > > On Mon, 2006-01-09 at 17:02 +0100, Oliver Neukum wrote:
> > > > > > > Am Montag, 9. Januar 2006 16:15 schrieb Lee Revell:
> > > > > > > > On Mon, 2006-01-09 at 15:28 +0100, Oliver Neukum wrote:
> > > > > > > > > Am Montag, 9. Januar 2006 15:18 schrieb Robert Hancock:
> > > > > > > > > > Yaroslav Rastrigin wrote:
> > > > > > > > > > > Well, I could find more or less reasonable explanation of this behaviour - different VM policies of two OSes and 
> > > > > > > > > > > strangely strong and persistent belief "Free RAM is a wasted RAM" among kernel devs. Free RAM is not a wasted RAM, its a memory waiting to be used ! 
> > > > > > > > > > > Whenever it is needed by apps I'm launching or working with. 
> > > > > > > > > > 
> > > > > > > > > > There is no different VM policy here, Windows behaves quite similarly. 
> > > > > > > > > > It does not leave memory around unused, it uses it for disk cache.
> > > > > > > > > 
> > > > > > > > > That doesn't mean that the rate of eviction is the same.
> > > > > > > > > Is it possible that read-ahead is not aggressive enough?
> > > > > > > > 
> > > > > > > > Enough for what?  What is the exact problem you are trying to solve?
> > > > > > > 
> > > > > > > Quicker application startup.
> > > > > > 
> > > > > > Why do you look to the kernel first?  The obvious explanation is that
> > > > > > Linux desktop apps are more bloated than their Windows counterparts.
> > > > > 
> > > > > It is the most efficient place. An improvement to the kernel will improve
> > > > > all starting times.
> > > > 
> > > > I think you'll get at most a 10% or 20% speedup by improving the kernel,
> > > > while some of these apps (think Nautilus vs Windows Explorer) will need
> > > > to be 1000% faster to seem reasonable to a Windows user.
> > > 
> > > That's easy: Just start nautilus, OOorg, Firefox, a java-vm and
> > > GNOME/KDE infrastructure at login time in the background (*eg* and
> > > mlockall() the more important ones so that the are surely in RAM) and
> > > "starting the app" is only a small program connecting to the respective
> > > process to get a fork() there (e.g. like the "-remote" parameter in the
> > > Mozilla family).
> > 
> > Have you tried this?  I suspect it still takes at least twice as long as
> > on windows.
> > 
> > For example on my system there was already a "nautilus" process but
> > "Places -> Home Folder" still took ~2 seconds to display anything, and
> > ~8 seconds to completely render the window and icons.  On Windows this
> > takes much less than a second.
> 
> Does the Windows Explorer draw icons based only on name and metadata?

Depends on the "View" of that directory and what you mean with
"metadata":
- There are view types that display such mini-icons and/or file details
  and/or contents for images.
- "Metadata" in the Windows-hell means "extension" which you get cheap
  with the filename anyways. There is no thing like "file" or a similar
  attempt to guess the type of contents automagically just for doing
  graphiocal "ls".

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Lee Revell]

On Mon, 2006-01-09 at 18:14 +0100, Olivier Galibert wrote:
> On Mon, Jan 09, 2006 at 05:53:35PM +0100, Oliver Neukum wrote:
> > Does the Windows Explorer draw icons based only on name and metadata?
> 
> >From what I can see it does icons on non-executable entirely based on
> the extension and nothing else on the first pass.  Executables are
> looked inside for an icon (and there seems to be cache effects at
> times, especially visible on the desktop).  Then for images a second
> pass generates icons depending on the contents (with, once again, a
> cache hidden somewhere).
> 
> Not a bad strategy, too.  Doing a file(1) on everything can only be
> slow given the random disk accesses it generates.  Maybe a file(1) as
> a _second_ pass would work.

Gack, does Nautilus really do file(1) on everything?  That's unspeakably
awful.

Lee

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Alan Cox]

On Llu, 2006-01-09 at 17:53 +0100, Oliver Neukum wrote:
> Does the Windows Explorer draw icons based only on name and metadata?

Sort of. It also plays tricks on the human by working out what icons are
visible and loading those first then filling in while the user thinks it
is ready

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Lee Revell]

On Mon, 2006-01-09 at 17:31 +0000, Alan Cox wrote:
> On Llu, 2006-01-09 at 17:53 +0100, Oliver Neukum wrote:
> > Does the Windows Explorer draw icons based only on name and metadata?
> 
> Sort of. It also plays tricks on the human by working out what icons are
> visible and loading those first then filling in while the user thinks it
> is ready
> 

See, I don't think that's a trick, isn't it just a form of demand
paging?  It seems silly to make the user wait to see the visible icons
until it's done rendering the hidden stuff.  Otherwise navigating the
file system speeds up and slows down based on how many files are in each
directory, even if the number of visible icons remains constant.

Do most Linux GUI apps really render everything without regard to what's
obscured and what's visible?

Lee

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Oliver Neukum]

Am Montag, 9. Januar 2006 18:31 schrieb Alan Cox:
> On Llu, 2006-01-09 at 17:53 +0100, Oliver Neukum wrote:
> > Does the Windows Explorer draw icons based only on name and metadata?
> 
> Sort of. It also plays tricks on the human by working out what icons are
> visible and loading those first then filling in while the user thinks it
> is ready

That's lazy evaluation. Actually quite clever.

	Regards
		Oliver

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Marcin Dalecki]

>
> Do most Linux GUI apps really render everything without regard to  
> what's
> obscured and what's visible?

Yes. Naive usage of backing store is right now pretty much the norm.  
However the
problem isn't the fact that the rendering occurs. The problem is that  
it occurs synchronously
and that composition is serializing with event handling.

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Anton Altaparmakov]

On Mon, 9 Jan 2006, Lee Revell wrote:
> On Mon, 2006-01-09 at 18:14 +0100, Olivier Galibert wrote:
> > On Mon, Jan 09, 2006 at 05:53:35PM +0100, Oliver Neukum wrote:
> > > Does the Windows Explorer draw icons based only on name and metadata?
> > 
> > >From what I can see it does icons on non-executable entirely based on
> > the extension and nothing else on the first pass.  Executables are
> > looked inside for an icon (and there seems to be cache effects at
> > times, especially visible on the desktop).  Then for images a second
> > pass generates icons depending on the contents (with, once again, a
> > cache hidden somewhere).
> > 
> > Not a bad strategy, too.  Doing a file(1) on everything can only be
> > slow given the random disk accesses it generates.  Maybe a file(1) as
> > a _second_ pass would work.
> 
> Gack, does Nautilus really do file(1) on everything?  That's unspeakably
> awful.

I believe it does a lot worse things.  It actually reads a picture file 
for example, then resizes the picture down to an icon sized one and then 
displays the icon, then on to the next file and for text files it 
displays the beginning of the text file in the icon, etc...  Totally 
unnecessary and brings total slowdown.  But of course as with Windows 
Explorer this is configurable and you can just use list view without 
icons...

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Alistair John Strachan]

On Monday 09 January 2006 16:41, Lee Revell wrote:
> On Mon, 2006-01-09 at 17:29 +0100, Bernd Petrovitsch wrote:
> > On Mon, 2006-01-09 at 11:19 -0500, Lee Revell wrote:
> > > On Mon, 2006-01-09 at 17:14 +0100, Oliver Neukum wrote:
> > > > Am Montag, 9. Januar 2006 17:04 schrieb Lee Revell:
> > > > > On Mon, 2006-01-09 at 17:02 +0100, Oliver Neukum wrote:
> > > > > > Am Montag, 9. Januar 2006 16:15 schrieb Lee Revell:
> > > > > > > On Mon, 2006-01-09 at 15:28 +0100, Oliver Neukum wrote:
> > > > > > > > Am Montag, 9. Januar 2006 15:18 schrieb Robert Hancock:
> > > > > > > > > Yaroslav Rastrigin wrote:
> > > > > > > > > > Well, I could find more or less reasonable explanation of
> > > > > > > > > > this behaviour - different VM policies of two OSes and
> > > > > > > > > > strangely strong and persistent belief "Free RAM is a
> > > > > > > > > > wasted RAM" among kernel devs. Free RAM is not a wasted
> > > > > > > > > > RAM, its a memory waiting to be used ! Whenever it is
> > > > > > > > > > needed by apps I'm launching or working with.
> > > > > > > > >
> > > > > > > > > There is no different VM policy here, Windows behaves quite
> > > > > > > > > similarly. It does not leave memory around unused, it uses
> > > > > > > > > it for disk cache.
> > > > > > > >
> > > > > > > > That doesn't mean that the rate of eviction is the same.
> > > > > > > > Is it possible that read-ahead is not aggressive enough?
> > > > > > >
> > > > > > > Enough for what?  What is the exact problem you are trying to
> > > > > > > solve?
> > > > > >
> > > > > > Quicker application startup.
> > > > >
> > > > > Why do you look to the kernel first?  The obvious explanation is
> > > > > that Linux desktop apps are more bloated than their Windows
> > > > > counterparts.
> > > >
> > > > It is the most efficient place. An improvement to the kernel will
> > > > improve all starting times.
> > >
> > > I think you'll get at most a 10% or 20% speedup by improving the
> > > kernel, while some of these apps (think Nautilus vs Windows Explorer)
> > > will need to be 1000% faster to seem reasonable to a Windows user.
> >
> > That's easy: Just start nautilus, OOorg, Firefox, a java-vm and
> > GNOME/KDE infrastructure at login time in the background (*eg* and
> > mlockall() the more important ones so that the are surely in RAM) and
> > "starting the app" is only a small program connecting to the respective
> > process to get a fork() there (e.g. like the "-remote" parameter in the
> > Mozilla family).
>
> Have you tried this?  I suspect it still takes at least twice as long as
> on windows.
>
> For example on my system there was already a "nautilus" process but
> "Places -> Home Folder" still took ~2 seconds to display anything, and
> ~8 seconds to completely render the window and icons.  On Windows this
> takes much less than a second.

Sounds like a broken configuration to me. Of course the usual array of 
hardware specs, configuration, filesystem, free RAM, etc. all play into this, 
but no KDE application, including the originally cited Kate, takes longer 
than 1s to start on this machine (1.6GHz P-M, 2MB L2, 400MHz FSB, 1GB 
PC2700).

Hell kfm is probably a hell of a lot more bloated than Nautilus and it's 
pretty fast to start first time (1-2s), and cached it's pretty much 
instantaneous (I'd say less than 400ms). Fast enough, no?

Sounds like the kernel is plenty fast.

-- 
Cheers,
Alistair.

'No sense being pessimistic, it probably wouldn't work anyway.'
Third year Computer Science undergraduate.
1F2 55 South Clerk Street, Edinburgh, UK.

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Lee Revell]

On Tue, 2006-01-10 at 01:44 +0000, Alistair John Strachan wrote:
> Sounds like a broken configuration to me. Of course the usual array of
> hardware specs, configuration, filesystem, free RAM, etc. all play
> into this, but no KDE application, including the originally cited
> Kate, takes longer than 1s to start on this machine (1.6GHz P-M, 2MB
> L2, 400MHz FSB, 1GB PC2700).
> 
> Hell kfm is probably a hell of a lot more bloated than Nautilus and
> it's pretty fast to start first time (1-2s), and cached it's pretty
> much instantaneous (I'd say less than 400ms). Fast enough, no?
> 

I don't think it's a broken configuration, just a slow machine (600MHz
VIA C3).  Windows XP screams compared to Linux on this thing.

Lee

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Jakob Oestergaard]

...
> > Hell kfm is probably a hell of a lot more bloated than Nautilus and
> > it's pretty fast to start first time (1-2s), and cached it's pretty
> > much instantaneous (I'd say less than 400ms). Fast enough, no?
> > 
> 
> I don't think it's a broken configuration, just a slow machine (600MHz
> VIA C3).  Windows XP screams compared to Linux on this thing.
> 

Guys... Apples and oranges and stuff.

My mother is running KDE on Linux now because (among other things)
windows explorer was unbearably slow for displaying folders with image
thumbnails  (very large images and lots of them).

Changing from an older 433MHz Celeron with too little memory to a 2.6
GHz P4 with a gig of memory was not enough. We tried quite a few things,
but I ended up giving KDE on Linux (Debian, not that it matters) a try.

I don't know Nautilus and I don't care - but I can say that there are
definitely situations in which KDE on Linux seriously and thoroughly
kicks MS butt both when it comes to simple usability and availability of
"good" software, and not least when it comes to the part of usability we
call "performance".  Getting the job done in time - and if something
takes a while to process, being able to do it in the background and
letting the user use the computer meanwhile.

This is not a kernel thing. There is proper desktop software out there -
go use it already  :)

My 0.02 Euro, for what it's worth,

-- 

 / jakob

File type by extension is evil (was Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time)

[Bernhard R. Link]

* Olivier Galibert <galibert@pobox.com> [060109 19:44]:
> >From what I can see it does icons on non-executable entirely based on
> the extension and nothing else on the first pass.
> 
> Not a bad strategy, too.  Doing a file(1) on everything can only be
> slow given the random disk accesses it generates.  Maybe a file(1) as
> a _second_ pass would work.

That may be a good strategy if you have user conditioned to all the
effects you get by this (i.e. if you only focus on Windows users and
want them provide with a system as broken as they know it) and programs
adopted to cope with the most ill effects (ever asked why some browsers
always foozle the name of downloaded files with some .html or the like?)

For everyone else looking at the file is the only sane way to know the type
of file.

	Bernhard R. Link

Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time

[Jesper Juhl]

On 1/10/06, Jakob Oestergaard <jakob@unthought.net> wrote:
> ...
> > > Hell kfm is probably a hell of a lot more bloated than Nautilus and
> > > it's pretty fast to start first time (1-2s), and cached it's pretty
> > > much instantaneous (I'd say less than 400ms). Fast enough, no?
> > >
> >
> > I don't think it's a broken configuration, just a slow machine (600MHz
> > VIA C3).  Windows XP screams compared to Linux on this thing.
> >
>
> Guys... Apples and oranges and stuff.
>
> My mother is running KDE on Linux now because (among other things)
> windows explorer was unbearably slow for displaying folders with image
> thumbnails  (very large images and lots of them).
>
> Changing from an older 433MHz Celeron with too little memory to a 2.6
> GHz P4 with a gig of memory was not enough. We tried quite a few things,
> but I ended up giving KDE on Linux (Debian, not that it matters) a try.
>
> I don't know Nautilus and I don't care - but I can say that there are
> definitely situations in which KDE on Linux seriously and thoroughly
> kicks MS butt both when it comes to simple usability and availability of
> "good" software, and not least when it comes to the part of usability we
> call "performance".  Getting the job done in time - and if something
> takes a while to process, being able to do it in the background and
> letting the user use the computer meanwhile.
>
> This is not a kernel thing. There is proper desktop software out there -
> go use it already  :)
>
> My 0.02 Euro, for what it's worth,
>
I have been using an 800MHz VIA C3 based box (256MB RAM) at work a
while back, running Slackware Linux 10.2 with XFCE as the desktop and
it's very usable - quite snappy infact. Even KDE runs resonably well
on that box once you turn off some of the eyecandy. I have no idea how
the box would run Win XP, but it's certainly quite fine as a Linux
workstation IMHO.

--
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: File type by extension is evil (was Re: Why the DOS has many ntfs read and write driver,but the linux can't for a long time)

[Lee Revell]

On Tue, 2006-01-10 at 11:32 +0100, Bernhard R. Link wrote:
> * Olivier Galibert <galibert@pobox.com> [060109 19:44]:
> > >From what I can see it does icons on non-executable entirely based on
> > the extension and nothing else on the first pass.
> > 
> > Not a bad strategy, too.  Doing a file(1) on everything can only be
> > slow given the random disk accesses it generates.  Maybe a file(1) as
> > a _second_ pass would work.
> 
> That may be a good strategy if you have user conditioned to all the
> effects you get by this (i.e. if you only focus on Windows users and
> want them provide with a system as broken as they know it) and programs
> adopted to cope with the most ill effects (ever asked why some browsers
> always foozle the name of downloaded files with some .html or the like?)
> 
> For everyone else looking at the file is the only sane way to know the type
> of file.

OK so it's prohibitively expensive to get the file type so this is not
something Nautilus should be doing to every file before even starting to
display the icons.

Lee