Re: 2 TB partition support

Re: 2 TB partition support

[Andi Kleen]

Bernd Schubert <Bernd.Schubert@tc.pci.uni-heidelberg.de> writes:

> Are 2TB possible with an unpatched 2.4.x 64bit-AMD64 kernel? The
> partion is supposed to be reiserfs. I read an about 2 years old
> discussion about this and Hans Reiser statet that the maximum size is
> about 2GB. Unfortunality I don't know what this 'about' depends on.
> Furthermore our server for this will be an Opteron and so perhaps this
> limit is much higher on 64bit systems.

In theory yes, but note that nobody tested the drivers for 64bit cleanness 
in block numbers. I would do careful testing first if your block driver supports 
>2TB.

-Andi

Re: 2 TB partition support

[Peter Chubb]

>>>>> "Andi" == Andi Kleen <ak@muc.de> writes:

Andi> Bernd Schubert <Bernd.Schubert@tc.pci.uni-heidelberg.de> writes:
>> Are 2TB possible with an unpatched 2.4.x 64bit-AMD64 kernel? The
>> partion is supposed to be reiserfs. I read an about 2 years old
>> discussion about this and Hans Reiser statet that the maximum size
>> is about 2GB. Unfortunality I don't know what this 'about' depends
>> on.  Furthermore our server for this will be an Opteron and so
>> perhaps this limit is much higher on 64bit systems.

Andi> In theory yes, but note that nobody tested the drivers for 64bit
Andi> cleanness in block numbers. I would do careful testing first if
Andi> your block driver supports
>> 2TB.

Has the kmalloc problem in Reiserfs gone away?  It used to be that the
limit for a Reiser filesystem was determined by how many pointers
could fit into a kmalloced chunk of memory; thus the 64-bit system
limit was half teh 32-bit system limit.

--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
The technical we do immediately,  the political takes *forever*

Re: 2 TB partition support

[Andi Kleen]

> Has the kmalloc problem in Reiserfs gone away?  It used to be that the
> limit for a Reiser filesystem was determined by how many pointers
> could fit into a kmalloced chunk of memory; thus the 64-bit system
> limit was half teh 32-bit system limit.

I don't know. i haven't tested reiserfs (or any other fs) with big file 
systems.

I was just talking about the theoretic limits in the block layer.

-Andi

Re: 2 TB partition support

[Hans Reiser]

Andi Kleen wrote:

>>Has the kmalloc problem in Reiserfs gone away?  It used to be that the
>>limit for a Reiser filesystem was determined by how many pointers
>>could fit into a kmalloced chunk of memory;
>>
?  I am not familiar with this....

>> thus the 64-bit system
>>limit was half teh 32-bit system limit.
>>    
>>
>
>I don't know. i haven't tested reiserfs (or any other fs) with big file 
>systems.
>
>I was just talking about the theoretic limits in the block layer.
>
>-Andi
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>
>  
>


-- 
Hans

Re: 2 TB partition support

[Peter Chubb]

>>>>> "Hans" == Hans Reiser <reiser@namesys.com> writes:

Hans> Andi Kleen wrote:
>>> Has the kmalloc problem in Reiserfs gone away?  It used to be that
>>> the limit for a Reiser filesystem was determined by how many
>>> pointers could fit into a kmalloced chunk of memory;
>>> 
Hans> ?  I am not familiar with this....

http://www.ussg.iu.edu/hypermail/linux/kernel/0207.0/0678.html

Peter C

RE: 2 TB partition support

[Xose Vazquez Perez]

Joseph Shamash wrote:

> I have searched without success for this driver.
> Qlogic tech support doesn't seem to know about it. 
> Can you lead me to a link or provide this driver?

2.6 driver http://sourceforge.net/projects/linux-qla2xxx/
2.4 driver http://download.qlogic.com/drivers/14612/qla2x00-v6.06.10-dist.tgz
--
HTML mails are going to trash automagically

2 TB partition support

[Joseph Shamash]

Hello,

I'm wondering if I can create a 2 TB partition using linux systems. If so,
do I need any special patches?

Thanks
____________________________________________
Joseph Shamash
    AVI(R)
  AVIstor(TM)
AVI Systems, Inc.
www.avistor.com
jshamash@avistor.com   Tel: (510) 644-1551   Fax: (510) 644-1991

1-888-883-2999 (toll free)
____________________________________________

2 TB partition support

[Peter Chubb]

>>>>> "Joseph" == Joseph Shamash <info@avistor.com> writes:

Joseph> Hello,

Joseph> I'm wondering if I can create a 2 TB partition using linux
Joseph> systems. If so, do I need any special patches?

Yes you can do it.

You need a 2.6 kernel.  And it's best to use something other than the
MSDOS partition format --- I suggest you use parted to create a GPT
partition table (which means compiling your kernel to understand that
format).

You didn't say what architecture you're running on.  If it's a 64-bit
system you don't have to do anything else.  If it's a 32-bit system,
then turn on CONFIG_LBD when you compile.

--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
The technical we do immediately,  the political takes *forever*

RE: 2 TB partition support

[Joseph Shamash]

Hello Peter,

Thank you for your quick reply.

Another Q.

What is the maximum partition size in TBs that 2.6 can handle?
What is the maximum file size?

Thanks,
Joe

-----Original Message-----
From: Peter Chubb [mailto:peterc@chubb.wattle.id.au]On Behalf Of Peter
Chubb
Sent: Monday, November 10, 2003 3:57 PM
To: Joseph Shamash
Cc: linux-kernel@vger.kernel.org
Subject: 2 TB partition support


>>>>> "Joseph" == Joseph Shamash <info@avistor.com> writes:

Joseph> Hello,

Joseph> I'm wondering if I can create a 2 TB partition using linux
Joseph> systems. If so, do I need any special patches?

Yes you can do it.

You need a 2.6 kernel.  And it's best to use something other than the
MSDOS partition format --- I suggest you use parted to create a GPT
partition table (which means compiling your kernel to understand that
format).

You didn't say what architecture you're running on.  If it's a 64-bit
system you don't have to do anything else.  If it's a 32-bit system,
then turn on CONFIG_LBD when you compile.

--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
The technical we do immediately,  the political takes *forever*

Re: 2 TB partition support

[Mike Fedyk]

On Mon, Nov 10, 2003 at 06:10:15PM -0800, Joseph Shamash wrote:
> Hello Peter,
> 
> Thank you for your quick reply.
> 
> Another Q.
> 
> What is the maximum partition size in TBs that 2.6 can handle?

Are you using hardware raid that shows the entire array like one disk that
should be partitioned, or do you want to use linux software raid?

If you're using hardware raid, you should consider what Peter said in his
message.

> What is the maximum file size?

That depends on what filesystem you want to use.  With ext2/3 it varies
between 16GB (with 1KB blocks) and 1 or 2 TB (with 4KB blocks) per file.

Other filesystems have similair limits.  There have been several
comparisons, and a quick google search should bring up a few.  Try "linux
filesystem comparison"

Mike

RE: 2 TB partition support

[Joseph Shamash]

Hello Peter,

Forgive another quick Q or two.

What is the maximum partition size for a patched 2.4.x kernel,
and where are those patches?

Thanks,
Joe

-----Original Message-----
From: Peter Chubb [mailto:peterc@chubb.wattle.id.au]On Behalf Of Peter
Chubb
Sent: Monday, November 10, 2003 3:57 PM
To: Joseph Shamash
Cc: linux-kernel@vger.kernel.org
Subject: 2 TB partition support


>>>>> "Joseph" == Joseph Shamash <info@avistor.com> writes:

Joseph> Hello,

Joseph> I'm wondering if I can create a 2 TB partition using linux
Joseph> systems. If so, do I need any special patches?

Yes you can do it.

You need a 2.6 kernel.  And it's best to use something other than the
MSDOS partition format --- I suggest you use parted to create a GPT
partition table (which means compiling your kernel to understand that
format).

You didn't say what architecture you're running on.  If it's a 64-bit
system you don't have to do anything else.  If it's a 32-bit system,
then turn on CONFIG_LBD when you compile.

--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
The technical we do immediately,  the political takes *forever*

Re: 2 TB partition support

[Mike Fedyk]

On Mon, Nov 10, 2003 at 06:12:06PM -0800, Joseph Shamash wrote:
> Hello Peter,
> 
> Forgive another quick Q or two.
> 
> What is the maximum partition size for a patched 2.4.x kernel,
> and where are those patches?

I believe it is now 16TB per block device in 2.6, and patched 2.4.

Re: 2 TB partition support

[Peter Chubb]

>>>>> "Mike" == Mike Fedyk <mfedyk@matchmail.com> writes:

> On Mon, Nov 10, 2003 at 06:12:06PM -0800, Joseph Shamash wrote:
>> 
>> What is the maximum partition size for a patched 2.4.x kernel, and
>> where are those patches?

Mike> I believe it is now 16TB per block device in 2.6, and patched
Mike> 2.4.

That's right for 32-bit systems with 4k pages.  For 64 bit systems the
limit is over 8 Exabytes.

You should note that software raid has smaller limits, as does the
LVM.  Also the 2.4 patches have seen *much* less testing than the 2.6
mainline (except possibly on the SGI Altix).

What exactly are you trying to do?

--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
The technical we do immediately,  the political takes *forever*

RE: 2 TB partition support

[Joseph Shamash]

>What exactly are you trying to do?

Doing testing in our lab trying to use 2.4.x and 2.6.
Trying to see how high a storage capacity can be supported
above 2TB, both in partitions and file sizes.

The limitation we have found in 2.4 is lack of 2TB support,
(using hardware raid with 2TB+ partitions).

The limitation we have found in 2.6 is lack FC HBA drivers which 
are needed to support large storage capacities.

Any thoughts?


-----Original Message-----
From: Peter Chubb [mailto:peterc@chubb.wattle.id.au]On Behalf Of Peter
Chubb
Sent: Monday, November 10, 2003 7:43 PM
To: Mike Fedyk
Cc: Joseph Shamash; Peter Chubb; linux-kernel@vger.kernel.org
Subject: Re: 2 TB partition support


>>>>> "Mike" == Mike Fedyk <mfedyk@matchmail.com> writes:

> On Mon, Nov 10, 2003 at 06:12:06PM -0800, Joseph Shamash wrote:
>> 
>> What is the maximum partition size for a patched 2.4.x kernel, and
>> where are those patches?

Mike> I believe it is now 16TB per block device in 2.6, and patched
Mike> 2.4.

That's right for 32-bit systems with 4k pages.  For 64 bit systems the
limit is over 8 Exabytes.

You should note that software raid has smaller limits, as does the
LVM.  Also the 2.4 patches have seen *much* less testing than the 2.6
mainline (except possibly on the SGI Altix).

What exactly are you trying to do?

--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
The technical we do immediately,  the political takes *forever*

Re: 2 TB partition support

[Patrick Mansfield]

On Mon, Nov 10, 2003 at 08:03:53PM -0800, Joseph Shamash wrote:

> The limitation we have found in 2.6 is lack FC HBA drivers which 
> are needed to support large storage capacities.
> 
> Any thoughts?

Please clarify "lack FC HBA drivers".

You mean no in kernel drivers? Yeh.

The qlogic (qla2xxx) driver is not in the kernel, but is available for use
with 2.6.

Martin Bligh included an emulex driver in his last 2.6 patch set.

-- Patrick Mansfield

RE: 2 TB partition support

[Joseph Shamash]

Hello Patrick,

>The qlogic (qla2xxx) driver is not in the kernel,
>but is available for use with 2.6.

I have searched without success for this driver.
Qlogic tech support doesn't seem to know about it. 
Can you lead me to a link or provide this driver?

Thanks,
Joe
 


-----Original Message-----
From: Patrick Mansfield [mailto:patmans@us.ibm.com]
Sent: Monday, November 10, 2003 9:30 PM
To: Joseph Shamash
Cc: Mike Fedyk; Peter Chubb; linux-kernel@vger.kernel.org
Subject: Re: 2 TB partition support


On Mon, Nov 10, 2003 at 08:03:53PM -0800, Joseph Shamash wrote:

> The limitation we have found in 2.6 is lack FC HBA drivers which 
> are needed to support large storage capacities.
> 
> Any thoughts?

Please clarify "lack FC HBA drivers".

You mean no in kernel drivers? Yeh.

The qlogic (qla2xxx) driver is not in the kernel, but is available for use
with 2.6.

Martin Bligh included an emulex driver in his last 2.6 patch set.

-- Patrick Mansfield

Re: 2 TB partition support

[Patrick Mansfield]

On Tue, Nov 11, 2003 at 01:21:42PM -0800, Joseph Shamash wrote:

> I have searched without success for this driver.
> Qlogic tech support doesn't seem to know about it. 
> Can you lead me to a link or provide this driver?
> 
> Thanks,
> Joe

Joe -

The driver is available off of this page:

http://sourceforge.net/projects/linux-qla2xxx/

-- Patrick Mansfield

Re: 2 TB partition support

[Randy.Dunlap]

On Tue, 11 Nov 2003 14:42:30 +1100 Peter Chubb <peter@chubb.wattle.id.au> wrote:

| >>>>> "Mike" == Mike Fedyk <mfedyk@matchmail.com> writes:
| 
| > On Mon, Nov 10, 2003 at 06:12:06PM -0800, Joseph Shamash wrote:
| >> 
| >> What is the maximum partition size for a patched 2.4.x kernel, and
| >> where are those patches?
| 
| Mike> I believe it is now 16TB per block device in 2.6, and patched
| Mike> 2.4.
| 
| That's right for 32-bit systems with 4k pages.  For 64 bit systems the
| limit is over 8 Exabytes.
| 
| You should note that software raid has smaller limits, as does the
| LVM.  Also the 2.4 patches have seen *much* less testing than the 2.6
| mainline (except possibly on the SGI Altix).
| 
| What exactly are you trying to do?


I made the table below for LinuxWorld Expo/Conference in Aug. 2002,
for Linux 2.4.x on 32-bit architectures, so it is a bit out of date,
but it might be helpful or useful.

--
~Randy
MOTD:  Always include version info.





Linux 2.4 filesystem limits on 32-bit architectures,
with 4 KB block sizes:


                     ext2/3fs    reiserfs     JFS       XFS#
max filesize:          4 TB&      16 TB$     16 TB$%  16 TB$
max filesystem size:  16 TB&      16 TB&     16 TB$   16 TB$
		                              4 PB&    8 EB&
kernel bldev limit:    2 TB        2 TB       2 TB     2 TB 


Notes:
#: all kernel limits
$: kernel limit
%: 4 KB pages
@: block device limit: 2 TB (or 1 TB if signed)
&: fs limit



Another look at ext2/3fs limits:

  Assumes using 4 KB block sizes on a 32-bit architecture
  (64-bit architecture isn't very limiting).

Largest limiting factor
           |
	   v
Smallest limiting factor


1.  64-bit API limit:  8 EiB

2.  kernel page cache index limit (32 bits) == filesystem block number:
    2^32 * 4 KB = 16 TiB

3.  ext2fs triple-indirect block limit:  4 TiB

4.  kernel block device limit (device sector number):  2 TiB
    (or 1 TiB if signed)

###

RE: 2 TB partition support

[Peter Chubb]

>>>>> "Joseph" == Joseph Shamash <info@avistor.com> writes:

Joseph> Hello Peter, Forgive another quick Q or two.

Joseph> What is the maximum partition size for a patched 2.4.x kernel,
Joseph> and where are those patches?

See http://www.gelato.unsw.edu.au/IA64wiki/LargeBlockDevices

I've only created a 2.4.20 patch; on my TODO list for the next
fortnight is to create a 2.4.23 patch, as we move towards a 2.4.23
release.

Petre C

RE: 2 TB partition support

[Joseph Shamash]

>I've only created a 2.4.20 patch;...

I seem to remember there was a bug for the 2.4.20 kernel. IIRC, it had
something to do with unmounting a filesystem and losing data, if the data
was still in the cache. Is this true? Can I find a patch for this, if it is
true?

Thanks



-----Original Message-----
From: Peter Chubb [mailto:peterc@chubb.wattle.id.au]On Behalf Of Peter
Chubb
Sent: Monday, November 10, 2003 6:39 PM
To: Joseph Shamash
Cc: Peter Chubb; linux-kernel@vger.kernel.org
Subject: RE: 2 TB partition support


>>>>> "Joseph" == Joseph Shamash <info@avistor.com> writes:

Joseph> Hello Peter, Forgive another quick Q or two.

Joseph> What is the maximum partition size for a patched 2.4.x kernel,
Joseph> and where are those patches?

See http://www.gelato.unsw.edu.au/IA64wiki/LargeBlockDevices

I've only created a 2.4.20 patch; on my TODO list for the next
fortnight is to create a 2.4.23 patch, as we move towards a 2.4.23
release.

Petre C

RE: 2 TB partition support

[Neil Brown]

On Monday November 10, info@avistor.com wrote:
> 
> >I've only created a 2.4.20 patch;...
> 
> I seem to remember there was a bug for the 2.4.20 kernel. IIRC, it had
> something to do with unmounting a filesystem and losing data, if the data
> was still in the cache. Is this true? Can I find a patch for this, if it is
> true?
> 

Only true (as far as I know) if using ext3 (and possibly other restrictions).

Patches at:
http://www.zipworld.com.au/~akpm/linux/ext3/

NeilBrown

Re: 2 TB partition support

[Bernd Schubert]

On Mon, Nov 10, 2003 at 06:12:06PM -0800, Joseph Shamash wrote:
> Hello Peter,
> 
> Forgive another quick Q or two.
> 
> What is the maximum partition size for a patched 2.4.x kernel,
> and where are those patches?
> 

Hello,

Are 2TB possible with an unpatched 2.4.x 64bit-AMD64 kernel? The
partion is supposed to be reiserfs. I read an about 2 years old
discussion about this and Hans Reiser statet that the maximum size is
about 2GB. Unfortunality I don't know what this 'about' depends on.
Furthermore our server for this will be an Opteron and so perhaps this
limit is much higher on 64bit systems.

I really wouldn't like to use the first 2.6.x releases on an important server like this. Also using hardly tested 2.4. patches are not really an option.

Thanks,
	Bernd

Re: 2 TB partition support

[Mike Fedyk]

On Wed, Nov 12, 2003 at 01:28:11AM +0100, Bernd Schubert wrote:
> On Mon, Nov 10, 2003 at 06:12:06PM -0800, Joseph Shamash wrote:
> > Hello Peter,
> > 
> > Forgive another quick Q or two.
> > 
> > What is the maximum partition size for a patched 2.4.x kernel,
> > and where are those patches?
> > 
> 
> Hello,
> 
> Are 2TB possible with an unpatched 2.4.x 64bit-AMD64 kernel? The
> partion is supposed to be reiserfs. I read an about 2 years old
> discussion about this and Hans Reiser statet that the maximum size is
> about 2GB. Unfortunality I don't know what this 'about' depends on.

That would refer to the Reiserfs 3.5 format, which is limited to 2GB max
file sizes.  If you use 2.4, and have never used a 2.2 reiserfs kernel, then
you probably have the 3.6 format that has a much larger limit.

Re: 2 TB partition support

[Peter Chubb]

>>>>> "Bernd" == Bernd Schubert <Bernd.Schubert@tc.pci.uni-heidelberg.de> writes:

Bernd> On Mon, Nov 10, 2003 at 06:12:06PM -0800, Joseph Shamash wrote:
>> Hello Peter,
>> 
>> Forgive another quick Q or two.
>> 
>> What is the maximum partition size for a patched 2.4.x kernel, and
>> where are those patches?
>> 

Bernd> Hello,

Bernd> Are 2TB possible with an unpatched 2.4.x 64bit-AMD64 kernel? 

On unpatched 2.4, the limit (depending on your driver) for a single
block device is either 2TB-1k or 1TB - 512b.

The 2.4 kernel keeps the block device sizes in an unsigned int, in 1k
units, so the maximum size is (2^32-1)*1k.

I forget which subsystem does it,but one of them tries to keep the
capacity of a disc in an unsigned int in 512byte units; if you're using
that subsystem, the macimum size you can use is (2^31-1)*512b

--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
The technical we do immediately,  the political takes *forever*

Re: 2 TB partition support

[Bernd Schubert]

> On unpatched 2.4, the limit (depending on your driver) for a single
> block device is either 2TB-1k or 1TB - 512b.
>
> The 2.4 kernel keeps the block device sizes in an unsigned int, in 1k
> units, so the maximum size is (2^32-1)*1k.
>
> I forget which subsystem does it,but one of them tries to keep the
> capacity of a disc in an unsigned int in 512byte units; if you're using
> that subsystem, the macimum size you can use is (2^31-1)*512b
>

Hello Peter,

 thanks for your help. Which driver doest this 2TB or 1TB-maximum blocksize 
size depend on? 
If we could get 2TB-1k, it would be great, since our raid will be 2.1TB and we 
plan to hardware-split it into 300MB+1800MB (hardware ide/scsi-system).

Thanks again, 
	Bernd

Re: 2 TB partition support

[Mike Fedyk]

On Fri, Nov 14, 2003 at 06:14:26PM +0100, Bernd Schubert wrote:
> > On unpatched 2.4, the limit (depending on your driver) for a single
> > block device is either 2TB-1k or 1TB - 512b.
> >
> > The 2.4 kernel keeps the block device sizes in an unsigned int, in 1k
> > units, so the maximum size is (2^32-1)*1k.
> >
> > I forget which subsystem does it,but one of them tries to keep the
> > capacity of a disc in an unsigned int in 512byte units; if you're using
> > that subsystem, the macimum size you can use is (2^31-1)*512b
> >
> 
> Hello Peter,
> 
>  thanks for your help. Which driver doest this 2TB or 1TB-maximum blocksize 
> size depend on? 

The hardware driver, ide or scsi subsystem, and the VFS all interact to
make the limitations on this.

It'd probably be best to post what kind of disk controllers you have, and
see what your limitations will be for them without the patch.  And even with
the patch, many of the hardware drivers may have lurking bugs for larger
block dev sizes.