linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Audio skips when RAM is ~full
@ 2003-11-01  5:31 Anthony DiSante
  2003-11-01  6:20 ` Willy Tarreau
  2003-11-01 18:08 ` Pavel Machek
  0 siblings, 2 replies; 8+ messages in thread
From: Anthony DiSante @ 2003-11-01  5:31 UTC (permalink / raw)
  To: linux-kernel

Hello,

I have a memory-related question regarding a Slackware system I'm running 
(kernel 2.4.22, compiled for i386, on a VIA Eden 600MHz processor, with 
256MB RAM).

First, the background info:

This system is primarily an mp3 player.  It boots directly to /bin/bash 
bypassing everything under /etc/rc.d, and just runs a small script that does 
modprobing and a few other miscellaneous things, then starts the main 
control script (which is Perl).

I have about 250 CDs encoded as mp3s on the 40gig drive, laid out in 
directories as /music/band/album/songs.mp3.  I also have a text file which 
contains one line for the full path to each album (so ~250 lines total). 
This file is used by my play-random-album function, which plays a random 
album each time I press the tick key.  When I do that, the songs from the 
randomly-chosen album become the current playlist, and the first song starts 
playing.

Now, the problem/question:

Each time I press the tick key, a new directory is read from the disk (and a 
song is played, and the directory contents are enqueued).  This causes the 
free memory (as reported by free) to drop by 1%, each time I press tick.  I 
often press it 5 or 10 or 20 times in a row until I find something I feel 
like listening to, and the free memory will drop by almost exactly 5 or 10 
or 20 percent by the time I settle on an album.  If the free memory was 
already very low (for example, when I do this after having already listened 
to an album or two), then it will often get down to 2% or 3% at this point, 
and stay there until I reboot.

This system has one of those awful on-board "ac'97" sound chips, which 
(AIUI) uses the system CPU to do much of the audio work instead of using 
dedicated audio chips for it.  When the free memory is less than about 10%, 
each song will "skip" when it first starts playing.  (It sounds like 1/2 or 
1 or 2 seconds of audio was deleted from the song; you don't hear silence, 
you hear the music at seconds 1, 2, and then 4, or just 2, 3, 4, ... 
skipping the very start, etc.  But it's always some portion of the first ~5 
seconds getting skipped.)

When the system is first booted, the free mem is around 90%, and this 
skipping doesn't happen.  As time goes on, the free mem drops, and skipping 
starts happening somewhere along the line; by the time it's down to ~10% 
free, every song skips when it starts.

The disk is a modern one using DMA.  And I even set up a ramdisk to try 
playing the songs from there (copying the "next" track into ramdisk while 
the current track is playing), but it still skips at the beginning of each 
track when the free mem is low enough.

My understanding/analysis/proposed solution:

The kernel is buffering the contents of each directory (album) that it reads 
(and also, mpg321 copies each mp3 file into RAM before playing it?).  I 
understand that the idea is to stuff as much into RAM as possible to reduce 
pagefile usage, and that the kernel will reclaim memory utilized by buffers 
if/when it needs to.  But apparently that isn't happening fast enough to 
allow a realtime process like music-playing to work skip-free on this system 
with this soundcard.  I think that if I could regularly forcibly dump the 
buffered stuff out of the RAM (dropping the used-RAM percentage down to, 
say, 10%, like at boot time) then this would make the skipping stop.

So... do I have a correct understanding of the problem, and a correct 
analysis of the kernel/mem issues that are related to it?  Is it possible to 
clear some of the RAM; if so, would that help?

Thanks,
Anthony

PS - you can see the system at:
http://nodivisions.com/tech/systems/musicbox/

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Audio skips when RAM is ~full
  2003-11-01  5:31 Audio skips when RAM is ~full Anthony DiSante
@ 2003-11-01  6:20 ` Willy Tarreau
  2003-11-01  6:34   ` Anthony DiSante
  2003-11-01 18:08 ` Pavel Machek
  1 sibling, 1 reply; 8+ messages in thread
From: Willy Tarreau @ 2003-11-01  6:20 UTC (permalink / raw)
  To: Anthony DiSante; +Cc: linux-kernel

Hello !

On Sat, Nov 01, 2003 at 12:31:15AM -0500, Anthony DiSante wrote:
> The kernel is buffering the contents of each directory (album) that it 
> reads (and also, mpg321 copies each mp3 file into RAM before playing it?).  
> I understand that the idea is to stuff as much into RAM as possible to 
> reduce pagefile usage, and that the kernel will reclaim memory utilized by 
> buffers if/when it needs to.  But apparently that isn't happening fast 
> enough to allow a realtime process like music-playing to work skip-free on 
> this system with this soundcard.  I think that if I could regularly 
> forcibly dump the buffered stuff out of the RAM (dropping the used-RAM 
> percentage down to, say, 10%, like at boot time) then this would make the 
> skipping stop.

1) are you certain that none of your programs (including mpg321) leaks
   memory ? As I understand it, it's really the cache which fills memory.
2) you can try to preload your file into the cache just before playing it :
   cp $file /dev/null ; mpg321 $file

> So... do I have a correct understanding of the problem, and a correct 
> analysis of the kernel/mem issues that are related to it?  Is it possible 
> to clear some of the RAM; if so, would that help?

Unless a leak happens somewhere in the kernel, sound driver, etc..., memory
consumed by programs is restored when your programs exit. The only part which
is not restored immediately is the cache.

BTW, there may be one other reason for your problem. Considering that you
scan your disk to find random albums, I think that the system updates all
directories access time after 5s, thus preventing your player from reading
an uncached file fast enough. You might be interested in mounting it with
the 'noatime,nodiratime' options in /etc/fstab.

You might also want to try 2.4.23pre9 which includes VM changes compared to
2.4.22, and seems quite stable to me.

Regards,
Willy


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Audio skips when RAM is ~full
  2003-11-01  6:20 ` Willy Tarreau
@ 2003-11-01  6:34   ` Anthony DiSante
  2003-11-01  7:19     ` Willy Tarreau
  2003-11-01  9:43     ` Maciej Zenczykowski
  0 siblings, 2 replies; 8+ messages in thread
From: Anthony DiSante @ 2003-11-01  6:34 UTC (permalink / raw)
  To: linux-kernel

Willy Tarreau wrote:
> Hello !
> 
> On Sat, Nov 01, 2003 at 12:31:15AM -0500, Anthony DiSante wrote:
> 
>>The kernel is buffering the contents of each directory (album) that it 
>>reads (and also, mpg321 copies each mp3 file into RAM before playing it?).  
>>I understand that the idea is to stuff as much into RAM as possible to 
>>reduce pagefile usage, and that the kernel will reclaim memory utilized by 
>>buffers if/when it needs to.  But apparently that isn't happening fast 
>>enough to allow a realtime process like music-playing to work skip-free on 
>>this system with this soundcard.  I think that if I could regularly 
>>forcibly dump the buffered stuff out of the RAM (dropping the used-RAM 
>>percentage down to, say, 10%, like at boot time) then this would make the 
>>skipping stop.
> 
> 
> 1) are you certain that none of your programs (including mpg321) leaks
>    memory ? As I understand it, it's really the cache which fills memory.

No, I'm not certain that mpg321 (or any other app, though there's 
next-to-nothing else running) is leak-free.

> 2) you can try to preload your file into the cache just before playing it :
>    cp $file /dev/null ; mpg321 $file

But wouldn't "cp $file /tmp/ramdisk0; mpg321 /tmp/ramdisk0/$file" accomplish 
the same thing?  (I've done that.)

>>So... do I have a correct understanding of the problem, and a correct 
>>analysis of the kernel/mem issues that are related to it?  Is it possible 
>>to clear some of the RAM; if so, would that help?
> 
> 
> Unless a leak happens somewhere in the kernel, sound driver, etc..., memory
> consumed by programs is restored when your programs exit. The only part which
> is not restored immediately is the cache.
> 
> BTW, there may be one other reason for your problem. Considering that you
> scan your disk to find random albums, I think that the system updates all
> directories access time after 5s, thus preventing your player from reading
> an uncached file fast enough. You might be interested in mounting it with
> the 'noatime,nodiratime' options in /etc/fstab.

Actually, I don't scan the disk to find random albums; I have a text file 
that contains a list of every album's full path, and I pick a random line 
from that file.  So only the selected album's directory gets scanned.  And 
the mp3 partition is mounted read-only (I should have mentioned that 
before), so the atimes shouldn't be getting written as it is.

> You might also want to try 2.4.23pre9 which includes VM changes compared to
> 2.4.22, and seems quite stable to me.

Maybe I will give that a try.

So I'm guessing that there isn't actually a way to manually move buffer-data 
out of RAM?

-Anthony
http://nodivisions.com/

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Audio skips when RAM is ~full
  2003-11-01  6:34   ` Anthony DiSante
@ 2003-11-01  7:19     ` Willy Tarreau
  2003-11-01 12:48       ` Anthony DiSante
  2003-11-01  9:43     ` Maciej Zenczykowski
  1 sibling, 1 reply; 8+ messages in thread
From: Willy Tarreau @ 2003-11-01  7:19 UTC (permalink / raw)
  To: Anthony DiSante; +Cc: linux-kernel

On Sat, Nov 01, 2003 at 01:34:12AM -0500, Anthony DiSante wrote:
> >2) you can try to preload your file into the cache just before playing it :
> >   cp $file /dev/null ; mpg321 $file
> 
> But wouldn't "cp $file /tmp/ramdisk0; mpg321 /tmp/ramdisk0/$file" 
> accomplish the same thing?  (I've done that.)

It's different because you need to reserve a fixed amount of ram for the
ramdisk while I was only using the caching features of the kernel as a
fast read-ahead mechanism. But I agree with you that the result should be
the same.

> Actually, I don't scan the disk to find random albums; I have a text file 
> that contains a list of every album's full path, and I pick a random line 
> from that file.  So only the selected album's directory gets scanned.  And 
> the mp3 partition is mounted read-only (I should have mentioned that 
> before), so the atimes shouldn't be getting written as it is.

OK, so it's not a disk IO problem at all. It's really related to the sound
driver it seems. Now that you say it, I remember having noticed skips on
my laptop with a via 82cxxx chip after tens of minutes playing. At first
I thought it was related to other activity on the system, but it did exactly
what you describe, play seconds 1, 2, then 5 without a hole between them.
There may be a problem with the way the audio buffer gets allocated or freed.

> So I'm guessing that there isn't actually a way to manually move 
> buffer-data out of RAM?

Yes, there is. I have a quick'n'dirty program which does exactly that.
Basically, you tell it how many kB you want to free, then it allocates
and uses that amount of memory, frees it and exits. Buffered data gets
flushed very quickly. I sometimes give it a try before starting to work
on large kernel trees, because it helps the entire directories to fit in
cache.

Here it is if you're interested. Don't start it without an argument, it
will try to allocate 4G !

Regards,
Willy

---
#include <stdio.h>

main(int argc, char **argv) {
  unsigned long int i,k=0, max;
  char *p;

  if (argc>1)
      max=atol(argv[1]);
  else
      max=0xffffffff;

  printf("Freeing %lu kB ram\n",max);
  while (((p=(char *)malloc(1048576))!=NULL) && (k+1024<=max)) {
    for (i=0;i<256;p[4096*i++]=0);
    k+=1024;
    fprintf(stderr,"\r%d kB allocated",k);
  }
  while (((p=(char *)malloc(65536))!=NULL) && (k+64<=max)) {
    for (i=0;i<16;p[4096*i++]=0);
    k+=64;
    fprintf(stderr,"\r%d kB allocated",k);
  }
  while (((p=(char *)malloc(1024))!=NULL) && (k+1<=max)) {
    for (i=0;i<16;p[64*i++]=0);
    k+=1;
    fprintf(stderr,"\r%d kB allocated",k);
  }
  fprintf(stderr,"\nMemory freed.\n");
  exit(0);
}


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Audio skips when RAM is ~full
  2003-11-01  6:34   ` Anthony DiSante
  2003-11-01  7:19     ` Willy Tarreau
@ 2003-11-01  9:43     ` Maciej Zenczykowski
  1 sibling, 0 replies; 8+ messages in thread
From: Maciej Zenczykowski @ 2003-11-01  9:43 UTC (permalink / raw)
  To: Anthony DiSante; +Cc: linux-kernel

> So I'm guessing that there isn't actually a way to manually move buffer-data 
> out of RAM?

Just the naive allocate X MB, set the first byte of every 4KB (PageSize) 
block to 1 and then free the memory - presto X MB free RAM.

Cheers,
MaZe.



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Audio skips when RAM is ~full
  2003-11-01  7:19     ` Willy Tarreau
@ 2003-11-01 12:48       ` Anthony DiSante
  2003-11-01 14:57         ` Jaroslav Kysela
  0 siblings, 1 reply; 8+ messages in thread
From: Anthony DiSante @ 2003-11-01 12:48 UTC (permalink / raw)
  To: linux-kernel

Willy Tarreau wrote:
>>Actually, I don't scan the disk to find random albums; I have a text file 
>>that contains a list of every album's full path, and I pick a random line 
>>from that file.  So only the selected album's directory gets scanned.  And 
>>the mp3 partition is mounted read-only (I should have mentioned that 
>>before), so the atimes shouldn't be getting written as it is.
> 
> 
> OK, so it's not a disk IO problem at all. It's really related to the sound
> driver it seems. Now that you say it, I remember having noticed skips on
> my laptop with a via 82cxxx chip after tens of minutes playing. At first
> I thought it was related to other activity on the system, but it did exactly
> what you describe, play seconds 1, 2, then 5 without a hole between them.
> There may be a problem with the way the audio buffer gets allocated or freed.

Yeah... in my experience, the whole ac'97 deal is pretty buggy (for one 
thing, with both ALSA and OSS drivers, output is distorted unless I keep the 
volume below ~70%...).  But as I said, the skipping only happens when the 
memory is full, or getting close to full.  If I select specific albums to 
play instead of using my random function, I can usually play a few albums 
and there aren't any skips, not until the memory gets pretty full.

>>So I'm guessing that there isn't actually a way to manually move 
>>buffer-data out of RAM?
> 
> 
> Yes, there is. I have a quick'n'dirty program which does exactly that.
> Basically, you tell it how many kB you want to free, then it allocates
> and uses that amount of memory, frees it and exits. Buffered data gets
> flushed very quickly. I sometimes give it a try before starting to work
> on large kernel trees, because it helps the entire directories to fit in
> cache.
> 
> Here it is if you're interested. Don't start it without an argument, it
> will try to allocate 4G !

4G!! (Note to self: s/ffffffff/0000ffff/...)  Thanks a lot, I'll give this a 
try.  It certainly works here on my home system.

-Anthony
http://nodivisions.com/

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Audio skips when RAM is ~full
  2003-11-01 12:48       ` Anthony DiSante
@ 2003-11-01 14:57         ` Jaroslav Kysela
  0 siblings, 0 replies; 8+ messages in thread
From: Jaroslav Kysela @ 2003-11-01 14:57 UTC (permalink / raw)
  To: Anthony DiSante; +Cc: linux-kernel

On Sat, 1 Nov 2003, Anthony DiSante wrote:

> Yeah... in my experience, the whole ac'97 deal is pretty buggy (for one
> thing, with both ALSA and OSS drivers, output is distorted unless I keep the
> volume below ~70%...).

It's ok. The AC97 codecs can apply gain (not only attenuation). Thus you
can overdrive your analog output and hear distortion. It's analog
behaviour not a bug.

						Jaroslav

-----
Jaroslav Kysela <perex@suse.cz>
Linux Kernel Sound Maintainer
ALSA Project, SuSE Labs

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Audio skips when RAM is ~full
  2003-11-01  5:31 Audio skips when RAM is ~full Anthony DiSante
  2003-11-01  6:20 ` Willy Tarreau
@ 2003-11-01 18:08 ` Pavel Machek
  1 sibling, 0 replies; 8+ messages in thread
From: Pavel Machek @ 2003-11-01 18:08 UTC (permalink / raw)
  To: Anthony DiSante; +Cc: linux-kernel

Hi!

Check your sound driver's sources . Perhaps it allocates GFP_ATOMIC
memory nd drops data on the floor if it is not available.

-- 
				Pavel
Written on sharp zaurus, because my Velo1 broke. If you have Velo you don't need...


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2003-11-11 19:21 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2003-11-01  5:31 Audio skips when RAM is ~full Anthony DiSante
2003-11-01  6:20 ` Willy Tarreau
2003-11-01  6:34   ` Anthony DiSante
2003-11-01  7:19     ` Willy Tarreau
2003-11-01 12:48       ` Anthony DiSante
2003-11-01 14:57         ` Jaroslav Kysela
2003-11-01  9:43     ` Maciej Zenczykowski
2003-11-01 18:08 ` Pavel Machek

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).