Big file support

Big file support

[Rick Stevens]

I'm not certain if this is the right place, but are there plans to
have big file support (files >2GB) anytime soon?  I ask, as we use
Linux to serve LOTS of streaming media and the logs for popular sites
often exceed 2GB.  I'd like to see the ability to handle at least 16GB
files, possibly more.

Please cc: me on any replies if possible.  I've been REALLY busy and
am finding it hard to keep up with l-k traffic.

Thanks!
----------------------------------------------------------------------
- Rick Stevens, SSE, VitalStream, Inc.      rstevens@vitalstream.com -
- 949-743-2010 (Voice)                    http://www.vitalstream.com -
-                                                                    -
-              Never eat anything larger than your head              -
----------------------------------------------------------------------

Re: Big file support

[Jon]

This support has been in ReiserFS for some time. On a SuSE system running
2.4.16 (their official kernel) you can issue: mkreiserfs -v2 /dev/sdaxxx and
there you go, LFS is in ver2.


Regards,

jon
----- Original Message -----
From: "Rick Stevens" <rstevens@vitalstream.com>
To: "Linux-Kernel" <linux-kernel@vger.kernel.org>
Sent: Wednesday, February 27, 2002 11:37 AM
Subject: Big file support


> I'm not certain if this is the right place, but are there plans to
> have big file support (files >2GB) anytime soon?  I ask, as we use
> Linux to serve LOTS of streaming media and the logs for popular sites
> often exceed 2GB.  I'd like to see the ability to handle at least 16GB
> files, possibly more.
>
> Please cc: me on any replies if possible.  I've been REALLY busy and
> am finding it hard to keep up with l-k traffic.
>
> Thanks!
> ----------------------------------------------------------------------
> - Rick Stevens, SSE, VitalStream, Inc.      rstevens@vitalstream.com -
> - 949-743-2010 (Voice)                    http://www.vitalstream.com -
> -                                                                    -
> -              Never eat anything larger than your head              -
> ----------------------------------------------------------------------
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Big file support

[Barubary]

A lot of the kernel supports big files already.  The real problem is the
fact that the primary Linux file system, ext3, does not.  If you use some
file system besides ext3, big files should work.

Linux already has API calls to read big files.

-- Barubary

----- Original Message -----
From: "Rick Stevens" <rstevens@vitalstream.com>
To: "Linux-Kernel" <linux-kernel@vger.kernel.org>
Sent: Wednesday, February 27, 2002 11:37 AM
Subject: Big file support


> I'm not certain if this is the right place, but are there plans to
> have big file support (files >2GB) anytime soon?  I ask, as we use
> Linux to serve LOTS of streaming media and the logs for popular sites
> often exceed 2GB.  I'd like to see the ability to handle at least 16GB
> files, possibly more.
>
> Please cc: me on any replies if possible.  I've been REALLY busy and
> am finding it hard to keep up with l-k traffic.
>
> Thanks!
> ----------------------------------------------------------------------
> - Rick Stevens, SSE, VitalStream, Inc.      rstevens@vitalstream.com -
> - 949-743-2010 (Voice)                    http://www.vitalstream.com -
> -                                                                    -
> -              Never eat anything larger than your head              -
> ----------------------------------------------------------------------
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Big file support

[Alan Cox]

> A lot of the kernel supports big files already.  The real problem is the
> fact that the primary Linux file system, ext3, does not.  If you use some
> file system besides ext3, big files should work.

This is incorrect information. Ext3 supports large files. Whoever told
you otherwise was wrong.

Re: Big file support

[Matti Aarnio]

On Wed, Feb 27, 2002 at 12:02:12PM -0800, Barubary wrote:
> A lot of the kernel supports big files already.  The real problem is the
> fact that the primary Linux file system, ext3, does not.  If you use some
> file system besides ext3, big files should work.

  Bullshit.   EXT2/EXT3 does support large files.  Has done so since
kernel 1.2 in fact, altough formely only at 64 bit machines.

Since 2.4 the kernel has been changed internally so that it supports
large files also at measly 32 bit thingies including i386...

There are several filesystems which are 64-bit/large-file supporting,
but also some which are inherently incapable to exceed 2G or 4G.

It looks like the LOOP driver lands in between -- it should be LFS
capable, but it isn't.

> Linux already has API calls to read big files.

Those were done for 2.4 kernel too.

> -- Barubary
> 
> ----- Original Message -----
> From: "Rick Stevens" <rstevens@vitalstream.com>
> To: "Linux-Kernel" <linux-kernel@vger.kernel.org>
> Sent: Wednesday, February 27, 2002 11:37 AM
> Subject: Big file support
> 
> 
> > I'm not certain if this is the right place, but are there plans to
> > have big file support (files >2GB) anytime soon?  I ask, as we use
> > Linux to serve LOTS of streaming media and the logs for popular sites
> > often exceed 2GB.  I'd like to see the ability to handle at least 16GB
> > files, possibly more.
> >
> > Please cc: me on any replies if possible.  I've been REALLY busy and
> > am finding it hard to keep up with l-k traffic.
> >
> > Thanks!
> > ----------------------------------------------------------------------
> > - Rick Stevens, SSE, VitalStream, Inc.      rstevens@vitalstream.com -
> > - 949-743-2010 (Voice)                    http://www.vitalstream.com -
> > -                                                                    -
> > -              Never eat anything larger than your head              -
> > ----------------------------------------------------------------------
> >
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Big file support\

[Chris Wedgwood]

On Wed, Feb 27, 2002 at 10:34:26PM +0200, Matti Aarnio wrote:

    It looks like the LOOP driver lands in between -- it should be LFS
    capable, but it isn't.

Really?

I used to use loop with large lop files all the time, I had to fix the
utils that mounted/attached the loop device though, as they would fail
without proper LFS build smarts.


   --cw

Re: Big file support

[Rik van Riel]

On Wed, 27 Feb 2002, Rick Stevens wrote:

> I'm not certain if this is the right place, but are there plans to
> have big file support (files >2GB) anytime soon?

This has been supported for well over a year.

Rik
-- 
"Linux holds advantages over the single-vendor commercial OS"
    -- Microsoft's "Competing with Linux" document

http://www.surriel.com/		http://distro.conectiva.com/

Re: Big file support

[Andreas Jaeger]

Rick Stevens <rstevens@vitalstream.com> writes:

> I'm not certain if this is the right place, but are there plans to
> have big file support (files >2GB) anytime soon?  I ask, as we use
> Linux to serve LOTS of streaming media and the logs for popular sites
> often exceed 2GB.  I'd like to see the ability to handle at least 16GB
> files, possibly more.

Check http://www.suse.de/~aj/linux_lfs.html,

Andreas
-- 
 Andreas Jaeger
  SuSE Labs aj@suse.de
   private aj@arthur.inka.de
    http://www.suse.de/~aj

Re: Big file support (emperical evidence)

[Stephen Samuel]

I agree -- and I was actually surprised.  On Tuesday I wrote
a script that created some huge files on an ext3 filesystem, expecting
it to die at 2GB, but it didn't die until I passed 8GB (and filled
the partition). (( redhat 7.2 ))

Alan Cox wrote:
>>A lot of the kernel supports big files already.  The real problem is the
>>fact that the primary Linux file system, ext3, does not.  If you use some
>>file system besides ext3, big files should work.
> 
> This is incorrect information. Ext3 supports large files. Whoever told
> you otherwise was wrong.


-- 
Stephen Samuel +1(604)876-0426                samuel@bcgreen.com
		   http://www.bcgreen.com/~samuel/
Powerful committed communication, reaching through fear, uncertainty and
doubt to touch the jewel within each person and bring it to life.

Re: Big file support

[Benjamin LaHaise]

On Wed, Feb 27, 2002 at 10:34:26PM +0200, Matti Aarnio wrote:
> There are several filesystems which are 64-bit/large-file supporting,
> but also some which are inherently incapable to exceed 2G or 4G.
> 
> It looks like the LOOP driver lands in between -- it should be LFS
> capable, but it isn't.

Loop is LFS capable.  I know that we shipped LFS enabled loop utilities 
for 7.2, probably 7.1 as well.  They were missed in the first batch of 
LFS conversions, and several distributions are lagging behind in this 
area.

		-ben

Re: Big file support

[Bill Davidsen]

On Wed, 27 Feb 2002, Rick Stevens wrote:

> I'm not certain if this is the right place, but are there plans to
> have big file support (files >2GB) anytime soon?  I ask, as we use
> Linux to serve LOTS of streaming media and the logs for popular sites
> often exceed 2GB.  I'd like to see the ability to handle at least 16GB
> files, possibly more.
> 
> Please cc: me on any replies if possible.  I've been REALLY busy and
> am finding it hard to keep up with l-k traffic.

You must be really behind, large file support has been in the current
kernel for ~14 months. Of course if your application isn't compiled with
LFS enabled it doesn't matter, or if it keeps offsets in long instead of
offset types...

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.