[linux-lvm] LILO configuration for LVM "boot" filesystem

[linux-lvm] LILO configuration for LVM "boot" filesystem

[Brian J. Murrell]

I know this is starting to become a LILO issue but I think that the
problem I have is really got to do with LILO and it's interacting with
LVM.  I have applied Andreas' patch, thanks Andreas.

When I try to boot with the LVM disk I get "LI" and that is it.  The
LILO docs say this about "LI":

This means the first-stage loader gained control; it thought it success-
fully loaded the second-stage loader; but it never got there.  This most
often occurs when the second-stage loader, '/boot/boot.b', is not load-
able using the BIOS.

Which would suggest to me that the mapping is not making it through
LVM correct me.  But I don't really know that much about how LILO
really works with the underlying device.

My current lilo.conf is:

boot=/dev/hdg
map=/boot/map
install=/boot/boot.b
vga=780
default=linux
keytable=/boot/us.klt
lba32
prompt
timeout=50
message=/boot/message

image=/boot/vmlinuz
    label=linux
    root=/dev/system/root
    initrd=/boot/initrd-lvm-2.4.5-1.1mdk.gz
    append=" hda=ide-scsi devfs=mount video=matrox:vesa:263"
    read-only

Anyone got any ideas or run into this before?

b.

-- 
Brian J. Murrell

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Ben Lutgens]

[-- Attachment #1: Type: text/plain, Size: 1841 bytes --]

On Mon, Jun 04, 2001 at 01:07:50AM -0700, Brian J. Murrell wrote:
>I know this is starting to become a LILO issue but I think that the
>problem I have is really got to do with LILO and it's interacting with
>LVM.  I have applied Andreas' patch, thanks Andreas.


Just curious, /boot only needs to be like 100 megs tops why would you want
that to be an LV. Isn't it more trouble than it's worth?

>
>When I try to boot with the LVM disk I get "LI" and that is it.  The
>LILO docs say this about "LI":
>
>This means the first-stage loader gained control; it thought it success-
>fully loaded the second-stage loader; but it never got there.  This most
>often occurs when the second-stage loader, '/boot/boot.b', is not load-
>able using the BIOS.
>
>Which would suggest to me that the mapping is not making it through
>LVM correct me.  But I don't really know that much about how LILO
>really works with the underlying device.
>
>My current lilo.conf is:
>
>boot=/dev/hdg
>map=/boot/map
>install=/boot/boot.b
>vga=780
>default=linux
>keytable=/boot/us.klt
>lba32
>prompt
>timeout=50
>message=/boot/message
>
>image=/boot/vmlinuz
>    label=linux
>    root=/dev/system/root
>    initrd=/boot/initrd-lvm-2.4.5-1.1mdk.gz
>    append=" hda=ide-scsi devfs=mount video=matrox:vesa:263"
>    read-only
>
>Anyone got any ideas or run into this before?
>
>b.
>
>-- 
>Brian J. Murrell
>_______________________________________________
>linux-lvm mailing list
>linux-lvm@sistina.com
>http://lists.sistina.com/mailman/listinfo/linux-lvm
>read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

-- 
Ben Lutgens		
Sistina Software Inc.	
MIS Geek / Gentoo Developer 
http://dolly-llama.org | http://www.gentoo.org/ | http://sistina.com/
Kernel panic: I have no root and I want to scream

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Andreas Dilger]

Brian writes:
> When I try to boot with the LVM disk I get "LI" and that is it.  The
> LILO docs say this about "LI":
> 
> This means the first-stage loader gained control; it thought it success-
> fully loaded the second-stage loader; but it never got there.  This most
> often occurs when the second-stage loader, '/boot/boot.b', is not load-
> able using the BIOS.

Are you sure that you can boot directly from /dev/hdg and/or the partition
that /dev/system/root lives on?  It may well be that the BIOS doesn't
understand this disk (I have never tried it myself).  Have you ever tried
booting from this setup without LVM?

On my LVM/LILO test system, I first made a regular partition for /boot,
and ensured that the BIOS could boot from there, and that I had all of
the right files, etc.  Then I backed the whole thing up, turned the
partition into a PV->VG->LV, restored the data and fixed up lilo.conf
to have the LV name instead of the partition name, and re-ran lilo.

> Which would suggest to me that the mapping is not making it through
> LVM correct me.  But I don't really know that much about how LILO
> really works with the underlying device.

I take it that this is not the other system we are talking about in which
you are moving disks around, is it?  That would complicate matters a bit
for trying to find the cause of this problem.

> My current lilo.conf is:
> 
> boot=/dev/hdg
> map=/boot/map
> install=/boot/boot.b
> vga=780
> default=linux
> keytable=/boot/us.klt
> lba32
> prompt
> timeout=50
> message=/boot/message
> 
> image=/boot/vmlinuz
>     label=linux
>     root=/dev/system/root
>     initrd=/boot/initrd-lvm-2.4.5-1.1mdk.gz
>     append=" hda=ide-scsi devfs=mount video=matrox:vesa:263"
>     read-only

Could you check that the output of "lilo -v5" produces reasonable block
numbers for your kernel image?  You can find out what "reasonable" is
by checking "lvdisplay -v /dev/system/root" output for PV location of
this LV, and then checking partition start offset via /proc/partitions.
Finally, you can use debugfs to list the filesystem blocks used by the
kernel image in that filesystem (via "stat file").

If you go the route of making a regular /boot partition first, you can
save the output of "lilo -v5" from before and after and compare.  There
should be a fixed offset between the partition and LV case, which should
match where your PE data starts (see "pvdata -PP /dev/hdX" and add pe_on_disk
base and size to get start of PE 0 on disk).

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Brian J. Murrell]

On Mon, Jun 04, 2001 at 09:21:42AM -0500, Ben Lutgens wrote:
> 
> Just curious, /boot only needs to be like 100 megs tops why would you want
> that to be an LV. Isn't it more trouble than it's worth?

If nothing else, how about "on principle"?  :-)  Seriously though, I
want to abandon screwing around with MS-DOS partitions and use LVs
instead.  They just make so much better sense than partitions.

b.

-- 
Brian J. Murrell

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Steven Lembark]

> > Just curious, /boot only needs to be like 100 megs tops why would you want
> > that to be an LV. Isn't it more trouble than it's worth?
> 
> If nothing else, how about "on principle"?  :-)  Seriously though, I
> want to abandon screwing around with MS-DOS partitions and use LVs
> instead.  They just make so much better sense than partitions.

It is seriously less hassle to have the root volume (with /boot
on it) as a simple file system.  Main point is that any error
in LVM will prevent you from booting to the point where you can
come up far enough to clean up the LVM system.  "Doable" and 
"make sense" aren't necessarly the same thing.  If you want to
use LVM for nearly everything create a 128MB root, 64MB swap and
few-hundred MB /var and use the 4th partition as pv00.  This 
will usually get you far enough up to fix anything that gets 
broken.  Until the PC BIOS mfr's include a sane boot system w/
console prompt that allows you to boot w/o LVM for damaged system
this will be much safer.

-- 
 Steven Lembark                                   2930 W. Palmer St.
                                                 Chicago, IL  60647
 lembark@wrkhors.com                                   800-762-1582

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Jack McKinney]

[-- Attachment #1: Type: text/plain, Size: 2665 bytes --]

Big Brother tells me that Steven Lembark wrote:
> 
> It is seriously less hassle to have the root volume (with /boot
> on it) as a simple file system.  Main point is that any error
> in LVM will prevent you from booting to the point where you can
> come up far enough to clean up the LVM system.  "Doable" and 
> "make sense" aren't necessarly the same thing.  If you want to
> use LVM for nearly everything create a 128MB root, 64MB swap and
> few-hundred MB /var and use the 4th partition as pv00.  This 
> will usually get you far enough up to fix anything that gets 
> broken.  Until the PC BIOS mfr's include a sane boot system w/
> console prompt that allows you to boot w/o LVM for damaged system
> this will be much safer.

    More importantly, it needs to be a lot more stable.  I will probably
wait on having a LV for root until Linux LVM is a robust as AIX's LVM.
Under AIX, all of your 'system' volumes (root,usr,var) are in rootvg.
You can add a disk to rootvg, and then issue pvmove to move all of the
blocks on the first disk to the new one, run bosboot (LILO, roughly
speaking) to make the second drive bootable, and then reduce the first
disk from the VG.  Thus, if your drives are hot-swappable, you can change
out a drive completely (even if your / is mounted on it and it has active
swap!) without every shutting down.
    Under Linux LVM, there are issues.  I did a demo to a local unix group
in which I created a volume and copied a bunch of files to it.  Then, I
started a pvmove command on that volume (while mounted) and copied files
into it while it was moving.  md5sums before and after verified that the
files were all intact.
    However, I did the same presentation on the same system a week later
to a local linux group, and one of the files got corrupted.  Moral: you
need to umount the FS in order to move the LV.  This means that you have
to reboot into single user mode in order to move the root volume.  Since
you have to umount an ext2 FS in order to move it, this means that the
root volume cannot use any of the benefits of LVM (except perhaps snapshots
and striping).  If you boot to single user mode, you could have moved/resized
root even as a regular partition.
    The only good reason to put root into an LV is that it is the cool
way to do things.  When LVM is more robust, it will even be the right
way to do things.

--
"There is no parameter that makes it impossible        Jack McKinney
     for you to perform still more excellently."       jackmc@lorentz.com
   -Mario Cuomo, on the lack of a clock in baseball    http://www.lorentz.com
1024D/D68F2C07 4096g/38AEF076

[-- Attachment #2: Type: application/pgp-signature, Size: 240 bytes --]

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Andreas Dilger]

Steven Lembar writes:
> It is seriously less hassle to have the root volume (with /boot
> on it) as a simple file system.  Main point is that any error
> in LVM will prevent you from booting to the point where you can
> come up far enough to clean up the LVM system.  "Doable" and 
> "make sense" aren't necessarly the same thing.

Yes, I think LVM is not quite at the stage yet where it is 100% robust
against crashes or errors during configuration changes.

> If you want to use LVM for nearly everything create a 128MB root,
> 64MB swap and few-hundred MB /var and use the 4th partition as pv00.

I don't know why you would want swap and /var as regular partitions.
Unless you have an incredibly small amount of RAM (i.e. 8MB or less)
you don't really need swap for booting.  Likewise, you don't need /var
right at boot time either.

In fact, on my 128MB laptop I don't use any swap at all, ever.  I first
turned it off when there were problems in the 2.4.x kernels with systems
going into livelock trying to flush swap pages, and never turned it on
again.  The only time I have memory problems is when I run silly filesystem
benchmarks that use 100k inodes, and the 2.4.5 kernel still doesn't flush
the inode slab cache properly.

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Andreas Dilger]

Jack McKinney writes:
>     Under Linux LVM, there are issues.  I did a demo to a local unix group
> in which I created a volume and copied a bunch of files to it.  Then, I
> started a pvmove command on that volume (while mounted) and copied files
> into it while it was moving.  md5sums before and after verified that the
> files were all intact.
>     However, I did the same presentation on the same system a week later
> to a local linux group, and one of the files got corrupted.  Moral: you
> need to umount the FS in order to move the LV.

FYI, this problem _should_ be fixed in CVS already, but the fix is more
recent than beta7, so it is likely that you hit a known bug.

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Brian J. Murrell]

On Tue, Jun 05, 2001 at 10:48:43AM -0500, Jack McKinney wrote:
> 
>     The only good reason to put root into an LV is that it is the cool
> way to do things.

Not only cool but sometimes necessary.  Sometimes root rides so close
to full that you cannot upgrade.  Then you start doing the "diskspace
shuffle" to get something you can upgrade.

Without wasting a ton of space when you initially install a system,
every system's root partition will hit the wall sooner or later.  That
is the unfortunate side effect of software evolution.

Anyway, I think we have exhausted the whys of doing it.  Now on to the
hows of doing it... I am still debugging.

b.

-- 
Brian J. Murrell

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Steven Lembark]

>     More importantly, it needs to be a lot more stable.  I will probably
> wait on having a LV for root until Linux LVM is a robust as AIX's LVM.
> Under AIX, all of your 'system' volumes (root,usr,var) are in rootvg.
> You can add a disk to rootvg, and then issue pvmove to move all of the
> blocks on the first disk to the new one, run bosboot (LILO, roughly
> speaking) to make the second drive bootable, and then reduce the first
> disk from the VG.  Thus, if your drives are hot-swappable, you can change
> out a drive completely (even if your / is mounted on it and it has active
> swap!) without every shutting down.

HP has a nice trick, the first 3 volumes have to be contiguous
at the start of the disk.  Net result is that if the VG gets
corrupted (e.g., one disk goes down and prevents quorum) you
can boot "-lm" which turns off the LVM service and leaves the
box looking like it was partitioned with boot, / & swap on line.

If LVM is used as a module and your rc.sysinit is sane (i.e., 
heavily stripped from all of the distribution standard ones)
then you can boot single user w/o locking up on a vgscan or 
vgchange.  If everything works, fine.  If not then at least
you can get going.

-- 
 Steven Lembark                                   2930 W. Palmer St.
                                                 Chicago, IL  60647
 lembark@wrkhors.com                                   800-762-1582

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Steven Lembark]

> Not only cool but sometimes necessary.  Sometimes root rides so close
> to full that you cannot upgrade.  Then you start doing the "diskspace
> shuffle" to get something you can upgrade.
> 
> Without wasting a ton of space when you initially install a system,
> every system's root partition will hit the wall sooner or later.  That
> is the unfortunate side effect of software evolution.

Use separate mount points for /var, /usr & /home.  This keeps
root down to mainly /etc, /dev, /bin, /sbin and /lib -- none of 
which is all that enormous.  Short of writing the File From Hell
to /tmp a 128MB root file system works well enough for me.  I do
put /usr & /opt on LVM but only because all the tools needed to
revive the system are in /bin, /sbin & /lvm on the root.

-- 
 Steven Lembark                                   2930 W. Palmer St.
                                                 Chicago, IL  60647
 lembark@wrkhors.com                                   800-762-1582

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Jack McKinney]

[-- Attachment #1: Type: text/plain, Size: 920 bytes --]

Big Brother tells me that Steven Lembark wrote:
> 
> Use separate mount points for /var, /usr & /home.  This keeps
> root down to mainly /etc, /dev, /bin, /sbin and /lib -- none of 
> which is all that enormous.  Short of writing the File From Hell
> to /tmp a 128MB root file system works well enough for me.  I do
> put /usr & /opt on LVM but only because all the tools needed to
> revive the system are in /bin, /sbin & /lvm on the root.

    After installation, do:

cd / ; mv tmp tmp.old ; ln -s /var/tmp .
reboot
rm -rf /tmp.old

    Now you don't have to worry about root filling up at all...

--
"Aluminum foil makes a wonderful hat and blocks     Jack McKinney
 out the government's mind-control rays."           http://www.lorentz.com
    Lt. Munch, The X-Files                          jackmc@lorentz.com
                                                    1024D/D68F2C07 4096g/38AEF076

[-- Attachment #2: Type: application/pgp-signature, Size: 240 bytes --]

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Jack McKinney]

[-- Attachment #1: Type: text/plain, Size: 970 bytes --]

Big Brother tells me that Jens Benecke wrote:
> >     After installation, do:
> > cd / ; mv tmp tmp.old ; ln -s /var/tmp .  reboot rm -rf /tmp.old
> >     Now you don't have to worry about root filling up at all...
> 
> er... I wouldn't do that (at least not on a Debian system). 
> 
> Usually /var/tmp is _assumed_ to be only root-writeable, so all sorts of
> daemons and programs running as root put their stuff there. This could open
> a number of security holes, when /var/tmp doesn't get treated as carefully
> as /tmp.

    Hmmm... I run Debian, and /var/tmp is certainly world-writable:

drwxrwxrwt    7 root     root         2048 Jun  7 07:56 /var/tmp


    It has been this way on every Debian system I have installed...

--
Whoever put the '.' next to the '/'       Jack McKinney
on a keyboard obviously never used        jackmc@lorentz.com
the -R option to /bin/chown               http://www.lorentz.com
1024D/D68F2C07 4096g/38AEF076

[-- Attachment #2: Type: application/pgp-signature, Size: 240 bytes --]

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Mario 'BitKoenig' Holbe]

On Thu, Jun 07, 2001 at 06:26:36PM +0200, Jens Benecke wrote:
> On Thu, Jun 07, 2001 at 10:43:02AM -0500, Jack McKinney wrote:
> > Big Brother tells me that Steven Lembark wrote:
> > > Use separate mount points for /var, /usr & /home.  This keeps root down
> > cd / ; mv tmp tmp.old ; ln -s /var/tmp .  reboot rm -rf /tmp.old

Windows user, eh? :)

> er... I wouldn't do that (at least not on a Debian system). 
> Usually /var/tmp is _assumed_ to be only root-writeable, so all sorts of

At least on a Debian system, /var/tmp is 1777 - just like /tmp.

But this 'solution' can cause some other problems in the boot
stage - /var isn't mounted there already, but maybe something
tries to write tmp files.

So - better create a /tmp filesystem as well - this doesnt harm,
because the /tmp mount point is a directory where processes can
write to, even when there's nothing mounted on.


regards,
-- 
Mario 'BitKoenig' Holbe <Mario.Holbe@RZ.TU-Ilmenau.DE>

So long and thanks for all the books.

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Jack McKinney]

[-- Attachment #1: Type: text/plain, Size: 755 bytes --]

Big Brother tells me that Jens Benecke wrote:
> On Thu, Jun 07, 2001 at 11:32:07AM -0500, Jack McKinney wrote:
>  
> >     Hmmm... I run Debian, and /var/tmp is certainly world-writable:
> 
> OK. sue me. This was a proposal in a "enhancing Debian security" paper that
> I found on the net (and implemented partially here). It seems not to be the
> default on new Debian installs.
> 
> Sorry.

    I'll have my lawyers contact your lawyers.   In the meantime, you can
help with the discovery procedures by tracking down that paper for us...
8-).

--
"I heard it was a lone gunman"                    Jack McKinney
   -X, The X Files                                jackmc@lorentz.com
1024D/D68F2C07 4096g/38AEF076  http://www.lorentz.com

[-- Attachment #2: Type: application/pgp-signature, Size: 240 bytes --]

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Michael Tokarev]

Steven Lembark wrote:
> 
[]
> Use separate mount points for /var, /usr & /home.  This keeps
> root down to mainly /etc, /dev, /bin, /sbin and /lib -- none of
> which is all that enormous.  Short of writing the File From Hell
> to /tmp a 128MB root file system works well enough for me.  I do
> put /usr & /opt on LVM but only because all the tools needed to
> revive the system are in /bin, /sbin & /lvm on the root.

Another thing to consider if you run 2.4 kernel -- mount tmpfs
on /tmp (and give some reasonable size restrictions).   This way,
/tmp works much faster, does not need to be cleaned on boot, and
will not eat root's space.  You should have reasonable swap space
it you plan to use it heavily.  Works very well here.

Regards,
 Michael.

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Steven Lembark]

> cd / ; mv tmp tmp.old ; ln -s /var/tmp .
> reboot
> rm -rf /tmp.old
> 
>     Now you don't have to worry about root filling up at all...

until you have to boot in single user w/o /var mounted and
the whole sysystem fries for lack of /tmp :-)

-- 
 Steven Lembark                                   2930 W. Palmer St.
                                                 Chicago, IL  60647
 lembark@wrkhors.com                                   800-762-1582

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Steven Lembark]

> > > Use separate mount points for /var, /usr & /home.  This keeps root down
> > > to mainly /etc, /dev, /bin, /sbin and /lib -- none of which is all that
> > > enormous.  Short of writing the File From Hell to /tmp a 128MB root
> > > file system works well enough for me.  I do put /usr & /opt on LVM but
> > > only because all the tools needed to revive the system are in /bin,
> > > /sbin & /lvm on the root.
> >     After installation, do:
> > cd / ; mv tmp tmp.old ; ln -s /var/tmp .  reboot rm -rf /tmp.old
> >     Now you don't have to worry about root filling up at all...
> 
> er... I wouldn't do that (at least not on a Debian system).
> 
> Usually /var/tmp is _assumed_ to be only root-writeable, so all sorts of
> daemons and programs running as root put their stuff there. This could open
> a number of security holes, when /var/tmp doesn't get treated as carefully
> as /tmp.

then debian butchered the SVR4 file system layout.  whole idea 
of having /boot and /var was to make sure that diskless systems
that nfs-ed everything else could count on having local storage
in /var and a minimal amount of anything required in /boot.

just about everything else on the planet assumes that /var/tmp
is a replacement for /tmp and should 01777...

-- 
 Steven Lembark                                   2930 W. Palmer St.
                                                 Chicago, IL  60647
 lembark@wrkhors.com                                   800-762-1582

RE: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Steve Wray]

yes, exactly.
Tell me, what is wrong with having /tmp as a mount point
for a LV?
(I'd suggest he got the symlink around the wrong way!
/var/tmp and /usr/tmp would be symlinks to /tmp...)


> -----Original Message-----
> From: linux-lvm-admin@sistina.com [mailto:linux-lvm-admin@sistina.com]On
> Behalf Of Steven Lembark
> Sent: Friday, June 08, 2001 9:19 AM
> To: linux-lvm@sistina.com
> Subject: Re: [linux-lvm] LILO configuration for LVM "boot" filesystem
> 
> 
> 
> > cd / ; mv tmp tmp.old ; ln -s /var/tmp .
> > reboot
> > rm -rf /tmp.old
> > 
> >     Now you don't have to worry about root filling up at all...
> 
> until you have to boot in single user w/o /var mounted and
> the whole sysystem fries for lack of /tmp :-)
> 
> -- 
>  Steven Lembark                                   2930 W. Palmer St.
>                                                  Chicago, IL  60647
>  lembark@wrkhors.com                                   800-762-1582
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html
> 

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Steven Lembark]

> Tell me, what is wrong with having /tmp as a mount point
> for a LV?

LV are generally slower than other types of storage.  Since the
boot partition is generally cyl 0 it's on the fastest storage
available (short of tmpfs).  

Another approach is to mirror-mount the /var/tmp and /tmp onto
a single volume.

-- 
 Steven Lembark                                   2930 W. Palmer St.
                                                 Chicago, IL  60647
 lembark@wrkhors.com                                   800-762-1582

RE: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Steve Wray]

> From: linux-lvm-admin@sistina.com [mailto:linux-lvm-admin@sistina.com]On
> Behalf Of Steven Lembark
> 
> > Tell me, what is wrong with having /tmp as a mount point
> > for a LV?
> 
> LV are generally slower than other types of storage.  Since the
> boot partition is generally cyl 0 it's on the fastest storage
> available (short of tmpfs).  

Proportionally, how much slower is it? 50%? 25%? 12%? 
less than 1% slower?
Does it really matter?


> 
> Another approach is to mirror-mount the /var/tmp and /tmp onto
> a single volume.

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Jack McKinney]

[-- Attachment #1: Type: text/plain, Size: 1244 bytes --]

Big Brother tells me that Steve Wray wrote:
> > 
> > LV are generally slower than other types of storage.  Since the
> > boot partition is generally cyl 0 it's on the fastest storage
> > available (short of tmpfs).  
> 
> Proportionally, how much slower is it? 50%? 25%? 12%? 
> less than 1% slower?

    Not even.  People don't realize how trivial LVM access actually are.
Suppose the OS has decided it needs to access block n of device foo. It
calls the driver to grab it.  If foo is sda1, then a read request is
scheduled (unless it is in the cache).  If foo is rootvg/usr_lv, then

1. It gets a pointer to the LE table (which it already has from the
device data when the read request is called in the lvm driver).
2. It executes a division, keeping the remainder (n divided by the
  LE size).
3. It looks up the quotient in the LE to get a real device name and
   offset.
4. It adds the remainder to the offset and calls the real low-level driver.

    In other words, every block read that goes through LVM adds basically
a division, a memory lookup and an addition.  The time it takes to transfer
the block from the drive is EONs compared to the time it takes to perform
a couple of arithmetical computations.


[-- Attachment #2: Type: application/pgp-signature, Size: 240 bytes --]

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Steven Lembark]

> Proportionally, how much slower is it? 50%? 25%? 12%?
> less than 1% slower?
> Does it really matter?

For things like /, /tmp or primary swap even a few % can
make a difference (especially on newer drives that use 
variable sector allocation).  This is something the system
hits frequently with quite a bit of data.

-- 
 Steven Lembark                                   2930 W. Palmer St.
                                                 Chicago, IL  60647
 lembark@wrkhors.com                                   800-762-1582

RE: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Steve Wray]

> From: linux-lvm-admin@sistina.com [mailto:linux-lvm-admin@sistina.com]On
> Behalf Of Jack McKinney
> 
> Big Brother tells me that Steve Wray wrote:
> > > 
> > > LV are generally slower than other types of storage.  Since the
> > > boot partition is generally cyl 0 it's on the fastest storage
> > > available (short of tmpfs).  
> > 
> > Proportionally, how much slower is it? 50%? 25%? 12%? 
> > less than 1% slower?
> 
[snip]
> 
>     In other words, every block read that goes through LVM adds basically
> a division, a memory lookup and an addition.  The time it takes 
> to transfer
> the block from the drive is EONs compared to the time it takes to perform
> a couple of arithmetical computations.

Unless the machine in question is an i386 with no math copro?
8-)

Sounds good to me; I was surprised someone raised a speed issue
for /tmp on a LV.
Are there any other (good) reasons to have /tmp (or anything else other
than /boot) on a 'real' partition rather than on LV?

How about swap on LV? Or does swap need to be used before the LV's
are accessible?

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Steven Lembark]

> Are there any other (good) reasons to have /tmp (or anything else other
> than /boot) on a 'real' partition rather than on LV?
> 
> How about swap on LV? Or does swap need to be used before the LV's
> are accessible?

Main issue is what will happen to the system if LVM gets broken.
The root volume (may not be the boot but usually is) and primary
swap are good bets for non-LVM only because you tend to need them
to repair LVM itself.

Depending on the disk, bus, cpu speed there may be an advantage
to having /tmp on non-LVM storage (only because you hit it so
hard; on HP-UX there is an advantage to having /tmp on mirrored
storage for speed :-).

-- 
 Steven Lembark                                   2930 W. Palmer St.
                                                 Chicago, IL  60647
 lembark@wrkhors.com                                   800-762-1582

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Mark van Walraven]

On Thu, Jun 07, 2001 at 10:43:02AM -0500, Jack McKinney wrote:
> cd / ; mv tmp tmp.old ; ln -s /var/tmp .
> reboot
> rm -rf /tmp.old
> 
>     Now you don't have to worry about root filling up at all...

/tmp is commonly emptied on boot, unlike /var/tmp.  Vi, for example,
stores it's recovery files in /var/tmp.

I prefer a separate partition for /tmp, to reduce the need to write to
the root fs and thereby lower the risk of damage to it in a crash.

Regards,

Mark.

RE: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Steve Wray]

> From: linux-lvm-admin@sistina.com [mailto:linux-lvm-admin@sistina.com]On
> Behalf Of Mark van Walraven
> 
> On Thu, Jun 07, 2001 at 10:43:02AM -0500, Jack McKinney wrote:
> > cd / ; mv tmp tmp.old ; ln -s /var/tmp .
> > reboot
> > rm -rf /tmp.old
> > 
> >     Now you don't have to worry about root filling up at all...
> 
> /tmp is commonly emptied on boot, unlike /var/tmp.  Vi, for example,
> stores it's recovery files in /var/tmp.
> 
> I prefer a separate partition for /tmp, to reduce the need to write to
> the root fs and thereby lower the risk of damage to it in a crash.

Yeah, I've been wondering about this whole discussion since
I thought it was "the done thing" to have /var and /tmp
on their own partitions (for precisely that reason).
I've been doing it that way since 1994 (when I moved up from
having my Linux on a 120 meg hard drive...)

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Russell Coker]

On Thursday 07 June 2001 23:52, Steve Wray wrote:
> > From: linux-lvm-admin@sistina.com
> > [mailto:linux-lvm-admin@sistina.com]On Behalf Of Steven Lembark
> >
> > > Tell me, what is wrong with having /tmp as a mount point
> > > for a LV?
> >
> > LV are generally slower than other types of storage.  Since the
> > boot partition is generally cyl 0 it's on the fastest storage
> > available (short of tmpfs).
>
> Proportionally, how much slower is it? 50%? 25%? 12%?
> less than 1% slower?
> Does it really matter?

See my Bonnie++ web page, particularly the ZCAV part.  The graph shows 
the start of the disk being ~80% faster than the end.  Try it out on your 
own drives!

This isn't an LVM issue, it's a disk geometry layout issue.

-- 
http://www.coker.com.au/bonnie++/     Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/       Postal SMTP/POP benchmark
http://www.coker.com.au/projects.html Projects I am working on
http://www.coker.com.au/~russell/     My home page

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Russell Coker]

On Thursday 07 June 2001 18:26, Jens Benecke wrote:
> >     After installation, do:
> > cd / ; mv tmp tmp.old ; ln -s /var/tmp .  reboot rm -rf /tmp.old
> >     Now you don't have to worry about root filling up at all...
>
> er... I wouldn't do that (at least not on a Debian system).
>
> Usually /var/tmp is _assumed_ to be only root-writeable, so all sorts
> of daemons and programs running as root put their stuff there. This
> could open a number of security holes, when /var/tmp doesn't get
> treated as carefully as /tmp.

I suggest using /var/tmp/tmp on a Debian system if you want to do this.  
That allows scripts that clean up /tmp to not clean /var/tmp (which may 
have things you want to last longer).

On Thursday 07 June 2001 23:08, Michael Tokarev wrote:
> Another thing to consider if you run 2.4 kernel -- mount tmpfs
> on /tmp (and give some reasonable size restrictions).   This way,
> /tmp works much faster, does not need to be cleaned on boot, and
> will not eat root's space.  You should have reasonable swap space
> it you plan to use it heavily.  Works very well here.

I tried that on 2.4.4 and I couldn't run the KDE.  It seems that tmpfs 
doesn't do all the things that a real FS does (yet).

On Thursday 07 June 2001 23:18, Steven Lembark wrote:
> > cd / ; mv tmp tmp.old ; ln -s /var/tmp .
> > reboot
> > rm -rf /tmp.old
> >
> >     Now you don't have to worry about root filling up at all...
>
> until you have to boot in single user w/o /var mounted and
> the whole sysystem fries for lack of /tmp :-)

Hasn't caused any great problems for me.  Although on systems with 
separate file systems for /var and /var/tmp it means that I need to run 
mount three times before I can edit a file in the root FS.


Another thing that no-one seems to be considering is the "--bind" option 
to recent versions of mount (only works on 2.4 kernels).


-- 
http://www.coker.com.au/bonnie++/     Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/       Postal SMTP/POP benchmark
http://www.coker.com.au/projects.html Projects I am working on
http://www.coker.com.au/~russell/     My home page

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Russell Coker]

On Friday 08 June 2001 00:23, Steven Lembark wrote:
> > Proportionally, how much slower is it? 50%? 25%? 12%?
> > less than 1% slower?
> > Does it really matter?
>
> For things like /, /tmp or primary swap even a few % can
> make a difference (especially on newer drives that use
> variable sector allocation).  This is something the system
> hits frequently with quite a bit of data.

However if you use two partitions at opposite ends of the drive then the 
seek times will kill performance...

-- 
http://www.coker.com.au/bonnie++/     Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/       Postal SMTP/POP benchmark
http://www.coker.com.au/projects.html Projects I am working on
http://www.coker.com.au/~russell/     My home page

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Mark van Walraven]

On Fri, Jun 08, 2001 at 03:30:36PM +0200, Jens Benecke wrote:
> On Fri, Jun 08, 2001 at 01:08:24AM +0400, Michael Tokarev wrote:
> > Another thing to consider if you run 2.4 kernel -- mount tmpfs on /tmp
> > (and give some reasonable size restrictions).   This way, /tmp works much
> > faster, does not need to be cleaned on boot, and will not eat root's
> > space.  You should have reasonable swap space it you plan to use it
> > heavily.  Works very well here.
> 
> tmpfs? I thought that existed only on Solaris, and the primary reason for
> tmpfs is that Solaris's UFS is so ssllooww compared to other FS (e.g.
> ext2, reiser, etc.)

It's in Linux 2.4 also.  It has the same unfortunate problem as on Solaris,
that once someone fills /tmp, you run out swap and daemons start dying.

Mark.

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Steven Lembark]

>> > Another thing to consider if you run 2.4 kernel -- mount tmpfs on /tmp
>> > (and give some reasonable size restrictions).   This way, /tmp works
>> > much faster, does not need to be cleaned on boot, and will not eat
>> > root's space.  You should have reasonable swap space it you plan to
>> > use it heavily.  Works very well here.
>>
>> tmpfs? I thought that existed only on Solaris, and the primary reason for
>> tmpfs is that Solaris's UFS is so ssllooww compared to other FS (e.g.
>> ext2, reiser, etc.)
>
> It's in Linux 2.4 also.  It has the same unfortunate problem as on
> Solaris, that once someone fills /tmp, you run out swap and daemons start
> dying.

Main reason for it on Solaris is systems w/ Large core (4GB+) who  (a) never
ever swap or noticably trim proc's and (b) got annoyed at wasiting 10+GB on
Sun's 2xcore rule for swapspace.  If you have  a4GB system w/ 8GB of 
[unused]
swap the odds of filling up /tmp are rather small.  If you have a 128MB 
system
with 10-20MB of swap in use it's a whole lot more likely.

 As always:  YMMV :-)

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Michael Tokarev]

Mark van Walraven wrote:
> 
> On Fri, Jun 08, 2001 at 03:30:36PM +0200, Jens Benecke wrote:
> > On Fri, Jun 08, 2001 at 01:08:24AM +0400, Michael Tokarev wrote:
> > > Another thing to consider if you run 2.4 kernel -- mount tmpfs on /tmp
> > > (and give some reasonable size restrictions).   This way, /tmp works much
> > > faster, does not need to be cleaned on boot, and will not eat root's
> > > space.  You should have reasonable swap space it you plan to use it
> > > heavily.  Works very well here.
> >
> > tmpfs? I thought that existed only on Solaris, and the primary reason for
> > tmpfs is that Solaris's UFS is so ssllooww compared to other FS (e.g.
> > ext2, reiser, etc.)
> 
> It's in Linux 2.4 also.  It has the same unfortunate problem as on Solaris,
> that once someone fills /tmp, you run out swap and daemons start dying.

Hey, do not allow users to fill it up! ;)  There is a mount option for
tmpfs on linux that allows you to specify max size of a filesystem to
create.

About tmpfs and a "real" filesystems.  No, ufs isn't so slow on solaris,
it is comparable with ext2, and faster sometimes (and slow other times),
it is hard to say which is better (I prefer ufs on solaris, as it is
more reliable with almost the same speed).  But with any real on-disk fs,
it is impossible to skip writing to disk (as fs designed to store data,
not to loose it), while tmpfs shurely can skip writing if there is
enouth memory, or with short files (ext2 will update directory at least,
maybe batching that update for many files, but it will not eliminate
update if a file will be created and deleted shortly).

BTW, this started to be offtopic... ;)

Regards,
 Michael.

Re: [linux-lvm] LILO configuration for LVM "boot" filesystem

[Russell Coker]

On Monday 11 June 2001 02:20, Michael Tokarev wrote:
> About tmpfs and a "real" filesystems.  No, ufs isn't so slow on
> solaris, it is comparable with ext2, and faster sometimes (and slow
> other times), it is hard to say which is better (I prefer ufs on

Last time I attempted to do a fair comparison I compared an Ultra 
Enterprise 2 with two UltraSPARC CPUs of ~200MHz speed, 256M of RAM, and 
a 9G SCSI hard drive with a Thinkpad 380XD with a 233MHz PentiumMMX, 96M 
of RAM and a 3.2G Toshiba IDE drive.

Ext2 on the Thinkpad beat UFS on the Sun in most tests.  ReiserFS on the 
Thinkpad beat tmpfs on the Sun for creating large numbers of files (tmpfs 
does not index directories).

This is off-topic.  Maybe we should migrate to the ReiserFS list where 
comparing FS performance is a topical matter?

-- 
http://www.coker.com.au/bonnie++/     Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/       Postal SMTP/POP benchmark
http://www.coker.com.au/projects.html Projects I am working on
http://www.coker.com.au/~russell/     My home page