Re: [linux-lvm] LVM, raw-io, devfs

Re: [linux-lvm] LVM, raw-io, devfs

[Daniel Whicker]

At 02:43 AM 10/3/99 , Harald Milz wrote:
>Drew Smith <drew@winterland.mainland.ab.ca> wrote:
>
> >       IBM's ADSM(*) (well, TSM now, thanks to Tivoli) uses raw logical
> > volumes on a fairly native level as storage pools, etc.  I'm curious,
> > having never looked into raw IO under Linux, whether it's the same
> > principles - and, that said, whether or not IBM will be porting the TSM
> > server to Linux?

   This is a bit off topic, buy hey.  ADSM (TSM) actually gives you the 
option of using either files or raw i/o.  I don't see that there would be 
any problems with a Linux port of the server.  They could simply disable 
the ability to use raw i/o for your disk storage pools, log volumes, and db 
volumes.  What may pose a problem is the amount of disk space required for 
a significant ADSM server.  If you're looking at a half terabyte or more of 
disks, you may run into the limit of Linux major/minor numbers for SCSI 
disks.  (Btw: I'm unclear on whether the devfs patch resolves this?)
									-Daniel
		------------------------------------------
		Daniel Whicker  (heimdall@mail.org)

Re: [linux-lvm] LVM, raw-io, devfs

[pamvdam]

On Mon, Oct 04, 1999 at 11:43:27AM +0100, Stephen C. Tweedie wrote:
> Hi,
> 
> On Sun, 3 Oct 1999 07:56:44 +0200, pamvdam@ramoth.xs4all.nl said:
> 
> > Oracle has always been using filesystems for it's datastorage. Only for
> > the parrallel server configuration they demand RAW IO. Like Sybase 11.9.x
> > under Linux it probably uses some O_SYNC flag to write to log, so the
> > write call only returns when the data has been really written to the
> > physical device. I don't think you'll see a performance enhancement when
> > using Oracle and RAW/IO. There's even a chance, due to the fact you mis
> > the buffer cache optimisations, that you'll see a performance
> > degradation.
> 
> It depends.  Raw IO uses less CPU --- much less in some cases.  It also
> avoids polluting memory with extra copies of data in the buffer cache.
> As a result, just measuring the pure disk bandwidth of raw IO versus
> buffered IO does not tell you the whole story about performance: you
> really need to measure the application's performance itself, including
> the effect of CPU and memory pressure.

You're right. I did oversee the buffercache pollution issue. Thanks
for pointing it out.

Best regards,

	Pascal



> 
> --Stephen

Re: [linux-lvm] LVM, raw-io, devfs

[Stephen C. Tweedie]

Hi,

On Sun, 3 Oct 1999 12:14:15 +0200, Adrien Farkas
<freddy@redneck.sk.oracle.com> said:

> Forget it, Linux port of Oracle DB doesn't support raw-io.

I believe that the current Oracle-8i release supports Linux raw IO out
of the box.

--Stephen

Re: [linux-lvm] LVM, raw-io, devfs

[Stephen C. Tweedie]

Hi,

On Sun, 3 Oct 1999 07:56:44 +0200, pamvdam@ramoth.xs4all.nl said:

> Oracle has always been using filesystems for it's datastorage. Only for
> the parrallel server configuration they demand RAW IO. Like Sybase 11.9.x
> under Linux it probably uses some O_SYNC flag to write to log, so the
> write call only returns when the data has been really written to the
> physical device. I don't think you'll see a performance enhancement when
> using Oracle and RAW/IO. There's even a chance, due to the fact you mis
> the buffer cache optimisations, that you'll see a performance
> degradation.

It depends.  Raw IO uses less CPU --- much less in some cases.  It also
avoids polluting memory with extra copies of data in the buffer cache.
As a result, just measuring the pure disk bandwidth of raw IO versus
buffered IO does not tell you the whole story about performance: you
really need to measure the application's performance itself, including
the effect of CPU and memory pressure.

--Stephen

Re: [linux-lvm] LVM, raw-io, devfs

[Adrien Farkas]

Fernando Dammous (ferdam@internetcom.com.br) wrote :

> Hi (again!),
> 
> From: Adrien Farkas <freddy@redneck.sk.oracle.com>
> > Forget it, Linux port of Oracle DB doesn't support raw-io.
> 
> I admit that I'm not a deep expert of Oracle, mainly regarding interaction
> with the I/O devices and with the operating system.
> 
> However, I have been finding in the documentation of Oracle for Linux,
> several references of utilization of RAW-DEVICES (among others).
> I'm not, here, wanting to create controversy, because it is dealt with a
> matter extra-LVM and being you a member of Oracle (I saw for your e-mail
> address freddy@sk.oracle.com), you must have knowledge of this subject. Then
> I ask:
>  What does mean the references about RAW-DEVICES in documentation?
> To illustrate, I separated some from :
> 
> Oracle8 Administrator's Reference for LINUX
> Release 8.0.5
> A66585-01

Oh, I'm sorry, Linux port of Oracle _does_ support raw io, but it depends on Oracle8 release, 8.0.5 does not, 8.0.5.1 (not released for public for free, afaik) and 8i (aka 8.1.5) both do. But definitely, I've never tried it. The last time I was interested in using raw partitions was in the time of 8.0.5, so prehaps that's why I still thought Oracle doesn't support them.

> <SNIP>
> - Tuning of I/O Loads across all Drives
> I/O loads are tuned across all drives, including drives storing Oracle data
> in raw devices
> <SNIP>
> - Raw Device Sizes
> Choose a small set of standard sizes for all raw devices that may be used to
> store Oracle database files.
> 
> In general, standardizing on a single size is recommended. If a single size
> is used, raw files can be moved from one partition to another safely. The
> size should be small enough so that a fairly large number can be created,
> but large enough to be convenient.
> 
> For example, a 2 GB drive could be divided into 10 partitions of 200 MB
> each-a good balance between size and number. Any tablespace using raw
> devices should stripe them across several drives. If possible, the striping
> should be done with a Logical Volume Manager."
> <SNIP>
> - Tuning the LINUX Buffer Cache Size
> To take full advantage of raw devices, adjust the size of the Oracle8 buffer
> cache and, if memory is limited, the LINUX buffer cache.
> 
> The LINUX buffer cache is provided by the operating system. It holds blocks
> of data in memory while they are being transferred from memory to disk, or
> vice versa.
> 
> The Oracle8 buffer cache is the area in memory that stores the Oracle
> database buffers. Since Oracle8 can use raw devices, it does not need to use
> the LINUX buffer cache.
> 
> When moving to raw devices, increase the size of the Oracle8 buffer cache.
> If the amount of memory on the system is limited, make a corresponding
> decrease in the LINUX buffer cache size.
> 
> The LINUX command vmstat may help you determine which buffer caches should
> be increased or decreased.
> <SNIP>
> ------------------------
> 
> Cheers and sorry for the long mail!
> Fernando
> 

-- 
                                             freddy

----==-- _                      Adrien "freddy" Farkas
---==---(_)__  __ ____  __       freddy@sk.oracle.com
--==---/ / _ \/ // /\ \/ /  We all know Linux is great... It
-=====/_/_//_/\___/ /_/\_\  does infinite loops in 5 seconds

Re: [linux-lvm] LVM, raw-io, devfs

[Fernando Dammous]

Hi (again!),

From: Adrien Farkas <freddy@redneck.sk.oracle.com>
> Forget it, Linux port of Oracle DB doesn't support raw-io.

I admit that I'm not a deep expert of Oracle, mainly regarding interaction
with the I/O devices and with the operating system.

However, I have been finding in the documentation of Oracle for Linux,
several references of utilization of RAW-DEVICES (among others).
I'm not, here, wanting to create controversy, because it is dealt with a
matter extra-LVM and being you a member of Oracle (I saw for your e-mail
address freddy@sk.oracle.com), you must have knowledge of this subject. Then
I ask:
 What does mean the references about RAW-DEVICES in documentation?
To illustrate, I separated some from :

Oracle8 Administrator's Reference for LINUX
Release 8.0.5
A66585-01

<SNIP>
- Tuning of I/O Loads across all Drives
I/O loads are tuned across all drives, including drives storing Oracle data
in raw devices
<SNIP>
- Raw Device Sizes
Choose a small set of standard sizes for all raw devices that may be used to
store Oracle database files.

In general, standardizing on a single size is recommended. If a single size
is used, raw files can be moved from one partition to another safely. The
size should be small enough so that a fairly large number can be created,
but large enough to be convenient.

For example, a 2 GB drive could be divided into 10 partitions of 200 MB
each-a good balance between size and number. Any tablespace using raw
devices should stripe them across several drives. If possible, the striping
should be done with a Logical Volume Manager."
<SNIP>
- Tuning the LINUX Buffer Cache Size
To take full advantage of raw devices, adjust the size of the Oracle8 buffer
cache and, if memory is limited, the LINUX buffer cache.

The LINUX buffer cache is provided by the operating system. It holds blocks
of data in memory while they are being transferred from memory to disk, or
vice versa.

The Oracle8 buffer cache is the area in memory that stores the Oracle
database buffers. Since Oracle8 can use raw devices, it does not need to use
the LINUX buffer cache.

When moving to raw devices, increase the size of the Oracle8 buffer cache.
If the amount of memory on the system is limited, make a corresponding
decrease in the LINUX buffer cache size.

The LINUX command vmstat may help you determine which buffer caches should
be increased or decreased.
<SNIP>
------------------------

Cheers and sorry for the long mail!
Fernando

Re: [linux-lvm] LVM, raw-io, devfs

[Adrien Farkas]

Fernando Dammous (ferdam@internetcom.com.br) wrote :

> HI Stephen and everybody,
> 
> > I've never tried the raw IO code on 2.0: let me know how it goes!  In
> > principle, there is no reason why raw IO on LVM should not work.
> >
> > --Stephen
> >
> 
> I prefer to update my kernel (to 2.3.18 or even to wait for the 2.4), before
> and then to use RAW-IO.
> I just hope I can simply alter Oracle to use RAW-IO instead of having
> reinstall it.

Forget it, Linux port of Oracle DB doesn't support raw-io.

> Using Oracle with LVM is already very good, with RAW-IO it will be better (I
> believe).
-- 
                                             freddy

----==-- _                      Adrien "freddy" Farkas
---==---(_)__  __ ____  __       freddy@sk.oracle.com
--==---/ / _ \/ // /\ \/ /  We all know Linux is great... It
-=====/_/_//_/\___/ /_/\_\  does infinite loops in 5 seconds

Re: [linux-lvm] LVM, raw-io, devfs

[Harald Milz]

Drew Smith <drew@winterland.mainland.ab.ca> wrote:

> 	IBM's ADSM(*) (well, TSM now, thanks to Tivoli) uses raw logical
> volumes on a fairly native level as storage pools, etc.  I'm curious,
> having never looked into raw IO under Linux, whether it's the same
> principles - and, that said, whether or not IBM will be porting the TSM
> server to Linux?  

I'm working on that one. Ummm... should we launch a petition? I'd be
willing to host a web page. 

-- 
Don't you feel more like you do now than you did when you came in?

Re: [linux-lvm] LVM, raw-io, devfs

[pamvdam]

On Sat, Oct 02, 1999 at 02:08:09PM -0300, Fernando Dammous wrote:
<stuff deleted>

> I prefer to update my kernel (to 2.3.18 or even to wait for the 2.4), before
> and then to use RAW-IO.
> I just hope I can simply alter Oracle to use RAW-IO instead of having
> reinstall it.
> Using Oracle with LVM is already very good, with RAW-IO it will be better (I
> believe).

I do question that. Sybase somehow needs (or better prefers) RAW-IO to ensure
it's structural integrity. If the buffer cache is between the data transfer
of the LOG and the physical device it is possible that not all of the log
that should be written out is written out, however Sybase thinks it has been
written to disk. RAW-IO/direct IO ensures that the LOG that has been ordered
to be written on disk will be written to disk. I would like to emphasize
that not using RAW-IO will not impose corrupting your database. It imposes
that there's a risk of loosing your data in case of a failure.

Oracle has always been using filesystems for it's datastorage. Only for
the parrallel server configuration they demand RAW IO. Like Sybase 11.9.x
under Linux it probably uses some O_SYNC flag to write to log, so the
write call only returns when the data has been really written to the
physical device. I don't think you'll see a performance enhancement when
using Oracle and RAW/IO. There's even a chance, due to the fact you mis
the buffer cache optimisations, that you'll see a performance degradation.

This is a common misunderstanding.

Best regards,

	Pascal van Dam



> In any way, I thank in advance for your interest (and of all people of here)
> in helping me in this case.
> 
> Thank you very much
> 
> Fernando
> 
> 

Re: [linux-lvm] LVM, raw-io, devfs

[Fernando Dammous]

HI Stephen and everybody,

> I've never tried the raw IO code on 2.0: let me know how it goes!  In
> principle, there is no reason why raw IO on LVM should not work.
>
> --Stephen
>

I prefer to update my kernel (to 2.3.18 or even to wait for the 2.4), before
and then to use RAW-IO.
I just hope I can simply alter Oracle to use RAW-IO instead of having
reinstall it.
Using Oracle with LVM is already very good, with RAW-IO it will be better (I
believe).
In any way, I thank in advance for your interest (and of all people of here)
in helping me in this case.

Thank you very much

Fernando

Re: [linux-lvm] LVM, raw-io, devfs

[Eric Smith]

I've been using the Linux ADSM v3 _client_ for some time but I've never
seen a server project.  Even the client is still listed as unsupported from
IBM although you can get it from them.  It's awesome...where's the
server?!?

--Eric









"Stephen C. Tweedie" wrote:

> Hi,
>
> On Fri, 01 Oct 1999 12:39:09 -0600, Drew Smith
> <drew@winterland.mainland.ab.ca> said:
>
> >       Now, I've just got to find a Linux-HSM(**) project to control my
> > optical jukebox. :)  Any ideas whether this exists?
>
> Not yet, but XFS has all of the support necessary for us to do a
> kernel DMAPI implementation, which would open up all sorts of HSM
> possibilities.
>
> --Stephen

Re: [linux-lvm] LVM, raw-io, devfs

[Stephen C. Tweedie]

Hi,

On Fri, 01 Oct 1999 12:39:09 -0600, Drew Smith
<drew@winterland.mainland.ab.ca> said:

> 	Now, I've just got to find a Linux-HSM(**) project to control my
> optical jukebox. :)  Any ideas whether this exists?

Not yet, but XFS has all of the support necessary for us to do a
kernel DMAPI implementation, which would open up all sorts of HSM
possibilities. 

--Stephen

Re: [linux-lvm] LVM, raw-io, devfs

[Luca Berra]

On Fri, Oct 01, 1999 at 12:39:09PM -0600, Drew Smith wrote:
> 	Now, I've just got to find a Linux-HSM(**) project to control my
> optical jukebox. :)  Any ideas whether this exists?

Ingo Molnar is doing HSM as part of linux-raid,
not working yet afaik

L.
-- 
Luca Berra -- bluca@comedia.it
    Communications Media & Services S.r.l.

Re: [linux-lvm] LVM, raw-io, devfs

[Drew Smith]

	Hi, just checking a fact. :)

	IBM's ADSM(*) (well, TSM now, thanks to Tivoli) uses raw logical
volumes on a fairly native level as storage pools, etc.  I'm curious,
having never looked into raw IO under Linux, whether it's the same
principles - and, that said, whether or not IBM will be porting the TSM
server to Linux?  

	Regardless, LVM is a huge jump in the right direction if we want IBM to
release more schtuff for Linux.  

	Now, I've just got to find a Linux-HSM(**) project to control my
optical jukebox. :)  Any ideas whether this exists?

(apologies for the footnotes, they're there for the benefit of people on
the list who don't know these acronyms)
*: ADSM: ADSTAR Distributed Storage Management.  A highly-cool
enterprise backup system, scheduling, etc - IBM's version of Veritas.
**: HSM: Heirarchial Storage Management.  Basically, take a tape library
or jukebox, make a database of everything on every cartridge, and mount
that database as a filesystem.  Copy something out of it, the library
goes and gets the appropriate media and feeds it to the drive.  With a
20-cart optical jukebox, this is basically a 20*1.3G filesystem (albiet
a fairly slow one, but with a script to copy the file into a temp space,
perfect as a low-use fileserver.  ADSM does this, but the cost is hefty,
to say the least.)

	Cheers,
	- Drew.
:wq

"Stephen C. Tweedie" wrote:
> 
> Hi,
> 
> On Thu, 30 Sep 1999 23:48:23 -0300, "Fernando Dammous"
> <ferdam@internetcom.com.br> said:
> 
> > Now, I would like to install the devfs and mainly the raw-io.
> 
> > Can I install them in a kernel 2.0.36, which is already patched with
> > LVM?
> 
> I've never tried the raw IO code on 2.0: let me know how it goes!  In
> principle, there is no reason why raw IO on LVM should not work.
> 
> --Stephen

Re: [linux-lvm] LVM, raw-io, devfs

[Stephen C. Tweedie]

Hi,

On Thu, 30 Sep 1999 23:48:23 -0300, "Fernando Dammous"
<ferdam@internetcom.com.br> said:

> Now, I would like to install the devfs and mainly the raw-io.   
  
> Can I install them in a kernel 2.0.36, which is already patched with
> LVM?

I've never tried the raw IO code on 2.0: let me know how it goes!  In
principle, there is no reason why raw IO on LVM should not work.

--Stephen

Re: [linux-lvm] LVM, raw-io, devfs

[Adrien Farkas]

Fernando Dammous (ferdam@internetcom.com.br) wrote :

> LVM 0.7 is installed (built into the kernel, which is still in the version 2.0.36) and working perfectly, with the root filesystem and swap in a LV (boot with initrd).  
>   
> Now, I would like to install the devfs and mainly the raw-io.   
>   
> Can I install them in a kernel 2.0.36, which is already patched with LVM?  
>   
> Should I apply the patches in which order (first devfs, later the raw-io, or vice versa)?  
>   
> I intend to install Oracle8 using raw-io.   

well, I'm still not sure whether Oracle software is compatible with raw-io patch in linux kernel, please let me know how are you doing with that.

> The devfs isn't so important for me (should it be ?).  

devfs is a very usable patch compatible with most of devices (including LVM) and software (I can only think of vmware that doesn't run with vmware). it saves a lot of inodes in /dev and contains only devices really present in the system and uses intelligent backward-compatible naming scheme (so you can still use inodes like /dev/sda etc) present in many modern UN*X flavors (like Solaris, e.g.). You should give it at lease a chance before saying you use it only because of raw-devices ;)

> If anyone knows about restrictions or some clues I would thank a lot.  

Regards,
-- 
                                             freddy

----==-- _                      Adrien "freddy" Farkas
---==---(_)__  __ ____  __       freddy@sk.oracle.com
--==---/ / _ \/ // /\ \/ /  We all know Linux is great... It
-=====/_/_//_/\___/ /_/\_\  does infinite loops in 5 seconds

[linux-lvm] LVM, raw-io, devfs

[Fernando Dammous]

[-- Attachment #1: Type: text/plain, Size: 788 bytes --]

Hi all,

 I would like help from the more experts in this matter, because I fear that I can in some way to lose or to corrupt my FS under LVM.  
  
LVM 0.7 is installed (built into the kernel, which is still in the version 2.0.36) and working perfectly, with the root filesystem and swap in a LV (boot with initrd).  
  
Now, I would like to install the devfs and mainly the raw-io.   
  
Can I install them in a kernel 2.0.36, which is already patched with LVM?  
  
Should I apply the patches in which order (first devfs, later the raw-io, or vice versa)?  
  
I intend to install Oracle8 using raw-io.   
The devfs isn't so important for me (should it be ?).  

If anyone knows about restrictions or some clues I would thank a lot.  

Thanks in advance

Fernando


[-- Attachment #2: Type: text/html, Size: 1411 bytes --]