* Re: File Limits
2011-01-18 8:55 File Limits Xavier Tarifa
@ 2011-01-18 8:54 ` Paul Crawford
2011-01-18 14:16 ` Xavier Tarifa
2011-01-18 17:35 ` Bryan J Smith
1 sibling, 1 reply; 10+ messages in thread
From: Paul Crawford @ 2011-01-18 8:54 UTC (permalink / raw)
To: Xavier Tarifa; +Cc: linux-msdos
Dear Xavier,
> So here's my problem: Until now I was using dosemu with ms-dos 6.22 to run an old database. A little while ago we had some big changes to the tables and they grew a lot.
> Yesterday I realized that we have a database file that is about to hit 2 Gb. I remembered that some time ago, we hit a bug on our program and it endlessly inserted rows on a table until it hit the 2 gb mark. Then the file either disappeared or became a 0 bytes file, I don't remember well.
> So I started reading and found that ms-dos 6.22 only supports FAT16 and apparently it only allows 2 gb volumes, and 2 gb files.
> So I thought of using ms-dos 7.10 which supports fat32 and would give us 4 gb files. But the same happens. The files disappear as soon as they hit the 2 gb mark. And when I execute the dir command, it always reports 2 gb of free disk space, which is odd.
>
> I don't know if this is a limitation of dosemu or I'm doing something wrong. Right now, I have about 140 users. On each $HOME/.dosemu/drive_c/ of the users there are symlinks to each file in /usr/share/dosemu/drive_z/ , where ms-dos is located. And I have the database files in /datos/ . I use lredir to make it appear as f: to every user.
> All of this is in the same ext3 partition.
>
> Any ideas?
The standard limits for 32-bit Linux programs is 2GB per file, even
though ext3 allows much, much more total space (and file size).
Have you tried using a 64-bit Linux and matching dosemu version?
I know it is not always as successful (e.g. I found the Microsoft C6
compiler works fine on Ubuntu 10.04 32-bit, but crashes/closes dosemu on
64-bit on the same hardware) but that would be my first suggestion to try.
Failing that, you may have to consider modifying the application to
split your data set or to store pointers to other data sets, etc, all of
which is a whole lot more trouble than just running it under dosemu!
Regards,
Paul
--
Dr. Paul S. Crawford
c/o Satellite Station
University of Dundee
Small's Wynd, Dundee, DD1 4HN
Email: psc@sat.dundee.ac.uk
Tel: +44 (0)1382 38 4687
The University of Dundee is a Scottish Registered Charity, No. SC015096
^ permalink raw reply [flat|nested] 10+ messages in thread
* File Limits
@ 2011-01-18 8:55 Xavier Tarifa
2011-01-18 8:54 ` Paul Crawford
2011-01-18 17:35 ` Bryan J Smith
0 siblings, 2 replies; 10+ messages in thread
From: Xavier Tarifa @ 2011-01-18 8:55 UTC (permalink / raw)
To: linux-msdos
So here's my problem: Until now I was using dosemu with ms-dos 6.22 to run an old database. A little while ago we had some big changes to the tables and they grew a lot.
Yesterday I realized that we have a database file that is about to hit 2 Gb. I remembered that some time ago, we hit a bug on our program and it endlessly inserted rows on a table until it hit the 2 gb mark. Then the file either disappeared or became a 0 bytes file, I don't remember well.
So I started reading and found that ms-dos 6.22 only supports FAT16 and apparently it only allows 2 gb volumes, and 2 gb files.
So I thought of using ms-dos 7.10 which supports fat32 and would give us 4 gb files. But the same happens. The files disappear as soon as they hit the 2 gb mark. And when I execute the dir command, it always reports 2 gb of free disk space, which is odd.
I don't know if this is a limitation of dosemu or I'm doing something wrong. Right now, I have about 140 users. On each $HOME/.dosemu/drive_c/ of the users there are symlinks to each file in /usr/share/dosemu/drive_z/ , where ms-dos is located. And I have the database files in /datos/ . I use lredir to make it appear as f: to every user.
All of this is in the same ext3 partition.
Any ideas?
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: File Limits
2011-01-18 8:54 ` Paul Crawford
@ 2011-01-18 14:16 ` Xavier Tarifa
2011-01-18 15:12 ` Paul Crawford
0 siblings, 1 reply; 10+ messages in thread
From: Xavier Tarifa @ 2011-01-18 14:16 UTC (permalink / raw)
To: Paul Crawford; +Cc: linux-msdos
> The standard limits for 32-bit Linux programs is 2GB per file, even though ext3 allows much, much more total space (and file size).
> Have you tried using a 64-bit Linux and matching dosemu version?
You're right, a 64-bits linux does work. I would have never thought that there was such a low limitation, though. It's the first program I can think of that I've found problems with big files (but the only other big files that I've used are mainly backups and vídeos so not a lot of apps involved).
Thanks a lot, that would have been probably the last reason I would think of.
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: File Limits
2011-01-18 14:16 ` Xavier Tarifa
@ 2011-01-18 15:12 ` Paul Crawford
2011-01-18 15:48 ` Bryan J Smith
0 siblings, 1 reply; 10+ messages in thread
From: Paul Crawford @ 2011-01-18 15:12 UTC (permalink / raw)
To: Xavier Tarifa; +Cc: linux-msdos
Dear Xavier,
Traditionally most UNIX / C language applications used a 32-bit signed
integer for file offset operations, hence the 2GB limit. This applied
even to DOS 16-bit applications as they also supported 32-bit 'long'
integers by special arithmetic.
You can get 32-bit applications that support >2GB because they are using
special arithmetic to implement 64-bit offsets, but that is of course
the 'natural' size for 64-bit applications.
At the time this was started 2GB was huge! When DOS was first available
in the early 1980s you could only get 10MB or so HDD. It was only around
1995 that you could get a consumer product HDD with 2GB capacity, let
alone consider single files of greater than this. And now my PC has 8GB
of memory!
Even today, the likes of fat32 has the problem that a large file (e.g.
video or backup archive, etc) can cause a write error if it exceeds 4GB.
Regards,
Paul
On 18/01/11 14:16, Xavier Tarifa wrote:
>> The standard limits for 32-bit Linux programs is 2GB per file, even
>> though ext3 allows much, much more total space (and file size).
>
>> Have you tried using a 64-bit Linux and matching dosemu version?
>
> You're right, a 64-bits linux does work. I would have never thought
> that there was such a low limitation, though. It's the first program
> I can think of that I've found problems with big files (but the only
> other big files that I've used are mainly backups and vídeos so not a
> lot of apps involved).
>
> Thanks a lot, that would have been probably the last reason I would
> think of.
--
Dr. Paul S. Crawford
Satellite Station
Dundee University
Small's Wynd, Dundee,
DD1 4HN, U.K.
Tel: +44 (0)1382 38 4687
Email: psc@sat.dundee.ac.uk
The University of Dundee is a Scottish Registered Charity, No. SC015096
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: File Limits
2011-01-18 15:12 ` Paul Crawford
@ 2011-01-18 15:48 ` Bryan J Smith
2011-01-18 16:39 ` Bryan J Smith
0 siblings, 1 reply; 10+ messages in thread
From: Bryan J Smith @ 2011-01-18 15:48 UTC (permalink / raw)
To: Paul Crawford, Xavier Tarifa; +Cc: linux-msdos
On 18/01/11 14:16, Xavier Tarifa wrote:
> The standard limits for 32-bit Linux programs is 2GB per file, even
> though ext3 allows much, much more total space (and file size).
Large File Support (LFS) on 32-bit POSIX implementations was addressed almost a
decade ago, including in Linux.
You must have three (3) things:
1) Kernel 2.4 (or a late 2.2 release with the backport, circa Alan Cox's
2.2.18+, some earlier)
2) GLibC 2.2 (or a late 2.1 release with the backport, circa 2.1.3+)
3) A program or build solution that implements the LFS macros for 32-bit
programs
For network file systems:
- NFS v3 or later (including SGI+Trond backports to Alan Cox's 2.2.18)
- Samba 2.2
Most programs written in the last decade should implement #3, and it's fairly
easy to do so in C code. Most interpreters also do. Unless you are explicitly
typecasting 32-bit ints, it's fairly easy to do #3. Any most any releases
starting with Red Hat Linux 7 and later do all of the above. I even had a
GLibC/kernel running on Red Hat Linux 6.2 from VA Linux that offered such.
However, for Windows itself, you must be using the new xFAT for >2GiB file
sizes, or >4GiB file sizes on NT kernels (a special call, not legacy compatible
-- NT offered a 4GiB capable FAT16 file system as well). Just having FAT32 is
not enough. xFAT is heavily patent encumbered though, and not Real Mode DOS
compatible.
On 18/01/11 14:16, Xavier Tarifa wrote:
> Have you tried using a 64-bit Linux and matching dosemu version?
I don't think that would work because x86-64 processors in Long Mode, as I
understand it, do not support Virtual86 Memory Models. Memory Models define
library/program compatibility.
Long Mode uses a 48-bit "flat" (again, "long") instruction pointer where the
segment register (16-bit) is concatenated with the offeset register (32-bit).
Protected386 and Virtual86 uses a 32-bit "normalized" instruction pointer where
the segment register (16-bit) is the segment where the offset register (32-bit)
is "flat." Real86 uses a 20-bit "normalized" instruction pointer where the
segment register (16-bit) is the segment where the offset register (16-bit) is
"flat."
So as I understand it ...
Protected386 is compatible with Virtual86 memory models and can execute Real86
programs.
Long Mode can execute Protected386, but cannot provide Virtual86 compatibility
and execute Real86 programs.
SIDE NOTE: This is also why x86-64 Long Mode cannot execute 386Enhanced models
(shunts processor between Protected386 and Real86).
Long Mode actually uses Physical Address Extensions (PAE) paging, 52-bit to be
exact. It can provide up to 52-bit/1PiB addressing with flat pages of up to
48-bit/256TiB sizes, although there are some flat 40-bit/1TiB and 36-bit/64GiB
limits in some structures (especially on Intel processors, and AMD before the
Processor 10h series).
PAE was originally introduced with the Pentium Pro and uses 36-bit, providing up
to 36-bit/64GiB addressing with flat pages up to 32-bit/4GiB. In other words,
there is a "hard" limitation at 52-bit/1PiB "paging" and 48-bit/256TiB "flat"
address models in the current x86-64 Long Mode memory model where it will break
even Protected386 (32-bit) compatibility.
It's all about memory models. I could be wrong though. Does DOSEmu provide
emulated Real86 compatibility when the processor is in x86-64 Long Mode?
-- Bryan
P.S. HyperVisors and virtualization will likely be the solution when 48/52-bit
limitations are hit in hardware. HyperVisors will no longer be optional, but
required and ubiquitous.
--
Bryan J Smith Professional, Technical Annoyance
Linked Profile: http://www.linkedin.com/in/bjsmith
----------------------------------------------------
UCF Basketball: AP #23, ESPN/USAToday #22, RPI #17
UCF Football: AP #21, BCS #25, ESPN/USAToday #20
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: File Limits
2011-01-18 15:48 ` Bryan J Smith
@ 2011-01-18 16:39 ` Bryan J Smith
0 siblings, 0 replies; 10+ messages in thread
From: Bryan J Smith @ 2011-01-18 16:39 UTC (permalink / raw)
To: Bryan J Smith, Paul Crawford, Xavier Tarifa; +Cc: linux-msdos
From: Bryan J Smith <b.j.smith@ieee.org>
> - Samba 2.2
Just want to make 1 clarification ...
All Real86 (DOS), Virtual86 (DOS, DOS/Win and NTVDM) and 386Enhanced (DOS/Win,
including through Windows Me) solutions still have client-side 2GiB file size
limitations. Samba 2.2+ just offers LFS support on 32-bit GNU/Linux platforms
for NT clients and other, true 32-bit CIFS/SMB clients (e.g., Linux itself).
Samba 2.2+ won't let you get around DOS, DOS/Win or NTVDM client limitations.
--
Bryan J Smith Professional, Technical Annoyance
Linked Profile: http://www.linkedin.com/in/bjsmith
----------------------------------------------------
UCF Basketball: AP #23, ESPN/USAToday #22, RPI #17
UCF Football: AP #21, BCS #25, ESPN/USAToday #20
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: File Limits
2011-01-18 8:55 File Limits Xavier Tarifa
2011-01-18 8:54 ` Paul Crawford
@ 2011-01-18 17:35 ` Bryan J Smith
2011-01-18 18:10 ` Xavier Tarifa
1 sibling, 1 reply; 10+ messages in thread
From: Bryan J Smith @ 2011-01-18 17:35 UTC (permalink / raw)
To: Xavier Tarifa, linux-msdos
All DOS (including DOS 7/Win 4 releases running in 386Enhanced mode -- Windows
95, 98, Me) only support 2GiB files IIRC. Even Real86 mode DOS 7 supports
FAT32, which allows bigger than 2GiB _file_systems_, but still has the 2GiB-1
(signed 32-bit int) _individual_file_ size limitation IIRC.
In fact, I don't think the 4GiB-1 (unsigned 32-bit int) support is in any
DOS/Win releases. I only think it is an option in NT kernels (FAT16 in 3.1,
3.5, 3.51 and 4.0, FAT16 and FAT32 in 5.0/2000, 5.1/XP-2003, 6.0-Vista and
6.1/7-2008). And any program that uses it _must_ know about the NT-only Win32
function calls that offer this.
I.e., the "Chicago" (Win9x) and NT/Win32 teams used _different_ DOS Int 21h and
Win32[s] function calls, the former completely ignoring the pre-existing of the
latter. "Chicago" _never_ supported the native NT/Win32 calls, hence why they
likely _never_ supported the extensions for 4GiB-1 support IIRC.[1]
One quick trip to TechNet and you'll note this is an issue in "bare metal"
DOS/Win and even NT. The legacy "Chicago" DOS Int 21h file services don't
handle growing beyond 2GiB-1 proper, _unless_ the NT/Win32 function extensions
are utilized. Unfortunately even NT 5.0 (Windows 2000) implements the "Chicago"
calls bug-for-bug for compatibility reasons.
I.e., your program is likely instigating this, and the OS is just letting it do
what it wants (and causing the issue).
-- Bryan
NOTES:
[1] Like many libraries in NT/Win32, things just "didn't work" or "work well"
under Win9x -- OpenGL being my personal favorite, and the whole "DirectMM -- aka
Direct [DOS] Memory Map" lineage now known as "DirectX". It wasn't until NT 5.0
(Windows 2000) that the NT/Win32 team started also supporting the
"Chicago"/Win32 DOS Int 21h changes.
----- Original Message ----
From: Xavier Tarifa <Xavier.Tarifa@adparts.com>
So here's my problem: Until now I was using dosemu with ms-dos 6.22 to run an
old database. A little while ago we had some big changes to the tables and they
grew a lot.
Yesterday I realized that we have a database file that is about to hit 2 Gb. I
remembered that some time ago, we hit a bug on our program and it endlessly
inserted rows on a table until it hit the 2 gb mark. Then the file either
disappeared or became a 0 bytes file, I don't remember well.
So I started reading and found that ms-dos 6.22 only supports FAT16 and
apparently it only allows 2 gb volumes, and 2 gb files.
So I thought of using ms-dos 7.10 which supports fat32 and would give us 4 gb
files. But the same happens. The files disappear as soon as they hit the 2 gb
mark. And when I execute the dir command, it always reports 2 gb of free disk
space, which is odd.
I don't know if this is a limitation of dosemu or I'm doing something wrong.
Right now, I have about 140 users. On each $HOME/.dosemu/drive_c/ of the users
there are symlinks to each file in /usr/share/dosemu/drive_z/ , where ms-dos is
located. And I have the database files in /datos/ . I use lredir to make it
appear as f: to every user.
All of this is in the same ext3 partition.
Any ideas?
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: File Limits
2011-01-18 17:35 ` Bryan J Smith
@ 2011-01-18 18:10 ` Xavier Tarifa
2011-01-18 19:18 ` Bryan J Smith
0 siblings, 1 reply; 10+ messages in thread
From: Xavier Tarifa @ 2011-01-18 18:10 UTC (permalink / raw)
To: Bryan J Smith; +Cc: linux-msdos
I just created a 3 gb file on a dos 7.10 and it worked (worked as in dir tells the correct filesize). I'll have to wait a day or two to test it against our database because I'm offsite, then I'll report back the results.
-----Mensaje original-----
De: linux-msdos-owner@vger.kernel.org [mailto:linux-msdos-owner@vger.kernel.org] En nombre de Bryan J Smith
Enviado el: martes, 18 de enero de 2011 18:35
Para: Xavier Tarifa; linux-msdos@vger.kernel.org
Asunto: Re: File Limits
All DOS (including DOS 7/Win 4 releases running in 386Enhanced mode -- Windows 95, 98, Me) only support 2GiB files IIRC. Even Real86 mode DOS 7 supports FAT32, which allows bigger than 2GiB _file_systems_, but still has the 2GiB-1 (signed 32-bit int) _individual_file_ size limitation IIRC.
In fact, I don't think the 4GiB-1 (unsigned 32-bit int) support is in any DOS/Win releases. I only think it is an option in NT kernels (FAT16 in 3.1, 3.5, 3.51 and 4.0, FAT16 and FAT32 in 5.0/2000, 5.1/XP-2003, 6.0-Vista and 6.1/7-2008). And any program that uses it _must_ know about the NT-only Win32 function calls that offer this.
I.e., the "Chicago" (Win9x) and NT/Win32 teams used _different_ DOS Int 21h and Win32[s] function calls, the former completely ignoring the pre-existing of the latter. "Chicago" _never_ supported the native NT/Win32 calls, hence why they likely _never_ supported the extensions for 4GiB-1 support IIRC.[1]
One quick trip to TechNet and you'll note this is an issue in "bare metal"
DOS/Win and even NT. The legacy "Chicago" DOS Int 21h file services don't handle growing beyond 2GiB-1 proper, _unless_ the NT/Win32 function extensions are utilized. Unfortunately even NT 5.0 (Windows 2000) implements the "Chicago"
calls bug-for-bug for compatibility reasons.
I.e., your program is likely instigating this, and the OS is just letting it do what it wants (and causing the issue).
-- Bryan
NOTES:
[1] Like many libraries in NT/Win32, things just "didn't work" or "work well"
under Win9x -- OpenGL being my personal favorite, and the whole "DirectMM -- aka Direct [DOS] Memory Map" lineage now known as "DirectX". It wasn't until NT 5.0 (Windows 2000) that the NT/Win32 team started also supporting the
"Chicago"/Win32 DOS Int 21h changes.
----- Original Message ----
From: Xavier Tarifa <Xavier.Tarifa@adparts.com>
So here's my problem: Until now I was using dosemu with ms-dos 6.22 to run an old database. A little while ago we had some big changes to the tables and they grew a lot.
Yesterday I realized that we have a database file that is about to hit 2 Gb. I remembered that some time ago, we hit a bug on our program and it endlessly inserted rows on a table until it hit the 2 gb mark. Then the file either disappeared or became a 0 bytes file, I don't remember well.
So I started reading and found that ms-dos 6.22 only supports FAT16 and apparently it only allows 2 gb volumes, and 2 gb files.
So I thought of using ms-dos 7.10 which supports fat32 and would give us 4 gb files. But the same happens. The files disappear as soon as they hit the 2 gb mark. And when I execute the dir command, it always reports 2 gb of free disk space, which is odd.
I don't know if this is a limitation of dosemu or I'm doing something wrong.
Right now, I have about 140 users. On each $HOME/.dosemu/drive_c/ of the users there are symlinks to each file in /usr/share/dosemu/drive_z/ , where ms-dos is located. And I have the database files in /datos/ . I use lredir to make it appear as f: to every user.
All of this is in the same ext3 partition.
Any ideas?
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: File Limits
2011-01-18 18:10 ` Xavier Tarifa
@ 2011-01-18 19:18 ` Bryan J Smith
2011-01-20 11:02 ` Xavier Tarifa
0 siblings, 1 reply; 10+ messages in thread
From: Bryan J Smith @ 2011-01-18 19:18 UTC (permalink / raw)
To: Xavier Tarifa; +Cc: linux-msdos
Then it's using the extended functions that legacy DOS Int 21h programs do not
understand. They may have merged in some NT/Win32 extensions in late Windows 98
SE and Me. They weren't in there for Windows 95, not even OEM 2 (MS-DOS 7.1)
when I tested them.
--
Bryan J Smith Professional, Technical Annoyance
Linked Profile: http://www.linkedin.com/in/bjsmith
----------------------------------------------------
UCF Basketball: AP #23, ESPN/USAToday #22, RPI #17
UCF Football: AP #21, BCS #25, ESPN/USAToday #20
----- Original Message ----
From: Xavier Tarifa <Xavier.Tarifa@adparts.com>
To: Bryan J Smith <b.j.smith@ieee.org>
Cc: "linux-msdos@vger.kernel.org" <linux-msdos@vger.kernel.org>
Sent: Tue, January 18, 2011 1:10:08 PM
Subject: RE: File Limits
I just created a 3 gb file on a dos 7.10 and it worked (worked as in dir tells
the correct filesize). I'll have to wait a day or two to test it against our
database because I'm offsite, then I'll report back the results.
-----Mensaje original-----
De: linux-msdos-owner@vger.kernel.org [mailto:linux-msdos-owner@vger.kernel.org]
En nombre de Bryan J Smith
Enviado el: martes, 18 de enero de 2011 18:35
Para: Xavier Tarifa; linux-msdos@vger.kernel.org
Asunto: Re: File Limits
All DOS (including DOS 7/Win 4 releases running in 386Enhanced mode -- Windows
95, 98, Me) only support 2GiB files IIRC. Even Real86 mode DOS 7 supports
FAT32, which allows bigger than 2GiB _file_systems_, but still has the 2GiB-1
(signed 32-bit int) _individual_file_ size limitation IIRC.
In fact, I don't think the 4GiB-1 (unsigned 32-bit int) support is in any
DOS/Win releases. I only think it is an option in NT kernels (FAT16 in 3.1,
3.5, 3.51 and 4.0, FAT16 and FAT32 in 5.0/2000, 5.1/XP-2003, 6.0-Vista and
6.1/7-2008). And any program that uses it _must_ know about the NT-only Win32
function calls that offer this.
I.e., the "Chicago" (Win9x) and NT/Win32 teams used _different_ DOS Int 21h and
Win32[s] function calls, the former completely ignoring the pre-existing of the
latter. "Chicago" _never_ supported the native NT/Win32 calls, hence why they
likely _never_ supported the extensions for 4GiB-1 support IIRC.[1]
One quick trip to TechNet and you'll note this is an issue in "bare metal"
DOS/Win and even NT. The legacy "Chicago" DOS Int 21h file services don't
handle growing beyond 2GiB-1 proper, _unless_ the NT/Win32 function extensions
are utilized. Unfortunately even NT 5.0 (Windows 2000) implements the "Chicago"
calls bug-for-bug for compatibility reasons.
I.e., your program is likely instigating this, and the OS is just letting it do
what it wants (and causing the issue).
-- Bryan
NOTES:
[1] Like many libraries in NT/Win32, things just "didn't work" or "work well"
under Win9x -- OpenGL being my personal favorite, and the whole "DirectMM -- aka
Direct [DOS] Memory Map" lineage now known as "DirectX". It wasn't until NT 5.0
(Windows 2000) that the NT/Win32 team started also supporting the
"Chicago"/Win32 DOS Int 21h changes.
----- Original Message ----
From: Xavier Tarifa <Xavier.Tarifa@adparts.com>
So here's my problem: Until now I was using dosemu with ms-dos 6.22 to run an
old database. A little while ago we had some big changes to the tables and they
grew a lot.
Yesterday I realized that we have a database file that is about to hit 2 Gb. I
remembered that some time ago, we hit a bug on our program and it endlessly
inserted rows on a table until it hit the 2 gb mark. Then the file either
disappeared or became a 0 bytes file, I don't remember well.
So I started reading and found that ms-dos 6.22 only supports FAT16 and
apparently it only allows 2 gb volumes, and 2 gb files.
So I thought of using ms-dos 7.10 which supports fat32 and would give us 4 gb
files. But the same happens. The files disappear as soon as they hit the 2 gb
mark. And when I execute the dir command, it always reports 2 gb of free disk
space, which is odd.
I don't know if this is a limitation of dosemu or I'm doing something wrong.
Right now, I have about 140 users. On each $HOME/.dosemu/drive_c/ of the users
there are symlinks to each file in /usr/share/dosemu/drive_z/ , where ms-dos is
located. And I have the database files in /datos/ . I use lredir to make it
appear as f: to every user.
All of this is in the same ext3 partition.
Any ideas?
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in the
body of a message to majordomo@vger.kernel.org More majordomo info at
http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: File Limits
2011-01-18 19:18 ` Bryan J Smith
@ 2011-01-20 11:02 ` Xavier Tarifa
0 siblings, 0 replies; 10+ messages in thread
From: Xavier Tarifa @ 2011-01-20 11:02 UTC (permalink / raw)
To: Bryan J Smith; +Cc: linux-msdos
The file límit seems to grow to 4 gb, but no luck running the application. On a 32 bits dosemu it works flawlessly, but on the 64 bits version it dies horribly. It seems to have problems reading/modifying/validating the license file, and only one user can use the db engine. As soon as another one logs in, it says that the max user count has been exceded. Strange thing is that the program that gives you info about the license file identifies it correclty as a 200 users license.
As soon as I replace the 64 bits dosemu with a 32 bits one it works ok again, but we're back to the 2 gb file size limit.
I'll try compiling it on my 32 bits machine with large file support, although I see that there are places where it's using int instead of off_t to seek files and this will need to be worked out.
-----Mensaje original-----
De: linux-msdos-owner@vger.kernel.org [mailto:linux-msdos-owner@vger.kernel.org] En nombre de Bryan J Smith
Enviado el: martes, 18 de enero de 2011 20:18
Para: Xavier Tarifa
CC: linux-msdos@vger.kernel.org
Asunto: Re: File Limits
Then it's using the extended functions that legacy DOS Int 21h programs do not understand. They may have merged in some NT/Win32 extensions in late Windows 98 SE and Me. They weren't in there for Windows 95, not even OEM 2 (MS-DOS 7.1) when I tested them.
--
Bryan J Smith Professional, Technical Annoyance
Linked Profile: http://www.linkedin.com/in/bjsmith
----------------------------------------------------
UCF Basketball: AP #23, ESPN/USAToday #22, RPI #17
UCF Football: AP #21, BCS #25, ESPN/USAToday #20
----- Original Message ----
From: Xavier Tarifa <Xavier.Tarifa@adparts.com>
To: Bryan J Smith <b.j.smith@ieee.org>
Cc: "linux-msdos@vger.kernel.org" <linux-msdos@vger.kernel.org>
Sent: Tue, January 18, 2011 1:10:08 PM
Subject: RE: File Limits
I just created a 3 gb file on a dos 7.10 and it worked (worked as in dir tells the correct filesize). I'll have to wait a day or two to test it against our database because I'm offsite, then I'll report back the results.
-----Mensaje original-----
De: linux-msdos-owner@vger.kernel.org [mailto:linux-msdos-owner@vger.kernel.org]
En nombre de Bryan J Smith
Enviado el: martes, 18 de enero de 2011 18:35
Para: Xavier Tarifa; linux-msdos@vger.kernel.org
Asunto: Re: File Limits
All DOS (including DOS 7/Win 4 releases running in 386Enhanced mode -- Windows 95, 98, Me) only support 2GiB files IIRC. Even Real86 mode DOS 7 supports FAT32, which allows bigger than 2GiB _file_systems_, but still has the 2GiB-1 (signed 32-bit int) _individual_file_ size limitation IIRC.
In fact, I don't think the 4GiB-1 (unsigned 32-bit int) support is in any DOS/Win releases. I only think it is an option in NT kernels (FAT16 in 3.1, 3.5, 3.51 and 4.0, FAT16 and FAT32 in 5.0/2000, 5.1/XP-2003, 6.0-Vista and 6.1/7-2008). And any program that uses it _must_ know about the NT-only Win32 function calls that offer this.
I.e., the "Chicago" (Win9x) and NT/Win32 teams used _different_ DOS Int 21h and Win32[s] function calls, the former completely ignoring the pre-existing of the latter. "Chicago" _never_ supported the native NT/Win32 calls, hence why they likely _never_ supported the extensions for 4GiB-1 support IIRC.[1]
One quick trip to TechNet and you'll note this is an issue in "bare metal"
DOS/Win and even NT. The legacy "Chicago" DOS Int 21h file services don't handle growing beyond 2GiB-1 proper, _unless_ the NT/Win32 function extensions are utilized. Unfortunately even NT 5.0 (Windows 2000) implements the "Chicago"
calls bug-for-bug for compatibility reasons.
I.e., your program is likely instigating this, and the OS is just letting it do what it wants (and causing the issue).
-- Bryan
NOTES:
[1] Like many libraries in NT/Win32, things just "didn't work" or "work well"
under Win9x -- OpenGL being my personal favorite, and the whole "DirectMM -- aka Direct [DOS] Memory Map" lineage now known as "DirectX". It wasn't until NT 5.0 (Windows 2000) that the NT/Win32 team started also supporting the
"Chicago"/Win32 DOS Int 21h changes.
----- Original Message ----
From: Xavier Tarifa <Xavier.Tarifa@adparts.com>
So here's my problem: Until now I was using dosemu with ms-dos 6.22 to run an old database. A little while ago we had some big changes to the tables and they grew a lot.
Yesterday I realized that we have a database file that is about to hit 2 Gb. I remembered that some time ago, we hit a bug on our program and it endlessly inserted rows on a table until it hit the 2 gb mark. Then the file either disappeared or became a 0 bytes file, I don't remember well.
So I started reading and found that ms-dos 6.22 only supports FAT16 and apparently it only allows 2 gb volumes, and 2 gb files.
So I thought of using ms-dos 7.10 which supports fat32 and would give us 4 gb files. But the same happens. The files disappear as soon as they hit the 2 gb mark. And when I execute the dir command, it always reports 2 gb of free disk space, which is odd.
I don't know if this is a limitation of dosemu or I'm doing something wrong.
Right now, I have about 140 users. On each $HOME/.dosemu/drive_c/ of the users there are symlinks to each file in /usr/share/dosemu/drive_z/ , where ms-dos is located. And I have the database files in /datos/ . I use lredir to make it appear as f: to every user.
All of this is in the same ext3 partition.
Any ideas?
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2011-01-20 11:02 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-01-18 8:55 File Limits Xavier Tarifa
2011-01-18 8:54 ` Paul Crawford
2011-01-18 14:16 ` Xavier Tarifa
2011-01-18 15:12 ` Paul Crawford
2011-01-18 15:48 ` Bryan J Smith
2011-01-18 16:39 ` Bryan J Smith
2011-01-18 17:35 ` Bryan J Smith
2011-01-18 18:10 ` Xavier Tarifa
2011-01-18 19:18 ` Bryan J Smith
2011-01-20 11:02 ` Xavier Tarifa
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.