From mboxrd@z Thu Jan 1 00:00:00 1970 From: Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?= Subject: Re: [Xen-devel] RE: How to: Improve Network Throughput of a XEN kernel? Date: Wed, 3 Mar 2010 23:41:32 +0200 Message-ID: <20100303214132.GN2761@reaktio.net> References: <20100303091844.GI2761@reaktio.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-users-bounces@lists.xensource.com Errors-To: xen-users-bounces@lists.xensource.com To: Mike Viau Cc: bderzhavets@yahoo.com, xen-devel@lists.xensource.com, xen-users@lists.xensource.com List-Id: xen-devel@lists.xenproject.org On Wed, Mar 03, 2010 at 10:44:38AM -0500, Mike Viau wrote: > > > > On Tue, Mar 02, 2010 at 07:50:59PM -0500, Mike Viau wrote: > > > >Any just first argument dummy=dummy should on kenel line. > > > > Read Wiki one more time. It works. > > > > > > I believe you that it does work. I was able to see (XEN) messages > coming from the hypervisor, but no matter how I changed the option I kept > getting a kernel fault. > > > > > > I feel like it was time to stop mucking around with getting Ubuntu and > the grub2 boot loader to work with XEN so I decided to install a headless > OpenSUSE 11.2 system. > > > > > > The openSUSE distribution supported Xen 3.4.1 tools, and had a > 2.6.31.12 xenified kernel. On this setup I experience identical network > performance. > > > > > > ~1MB/s - w/ xen kernel > > > ~4.5 MB/s - w/ non-xen kernel > > > > > > I use the e1000e kernel module driver for my Ethernet card. > > > > > > Is this typical Dom0 (or Xen Host) network performance with a xenified > kernel or is just in my two setups that one encounters such a network > performance hit to the Dom0? > > > > > > > No, it's not typical. > > > > How are you measuring the performance? In a gigabit network you should > be able > > to push around 110 MB/sec.. 4.5 MB/sec sounds really slow. > > I am using Fast Ethernet (100MB). What sort of throughput should I expect > becasue I also believe ~1MB is too slow, even just to/from the Dom0, > without any network emulation to the DomUs. > 100 Mbit ethernet can transfer around 11 MB/sec. > I simply used SCP, HTTP, and FTP file transfer protocols on the Xen host > to another computer on the LAN. The ~4.5MB/s bottleneck is the hard drive > on the other computer so I suspect the true bandwidth would be higher, but > getting past the ~1MB/s roadblock is my major concern. > Try using normal FTP (without encryption), or iperf. Or wget to /dev/null, like: wget -O /dev/null http://testserver/file > I have tried two linux distros and both forward ported xenified and > parvirtualized xen kernel. > > Does the Xen hypervisor have any effect on the network throughput? Would > updating the Intel E1000E kernel module driver affect the network > throughput? > Shouldn't be big difference, when you measure from dom0. -- Pasi > -M > > > > > -- Pasi > > > > > -M > > > > > > --- On Mon, 3/1/10, Mike Viau wrote: > > > > > > From: Mike Viau > > > Subject: RE: [Xen-devel] RE: [Xen-users] How to: Improve Network > Throughput of a XEN kernel? > > > To: bderzhavets@yahoo.com > > > Cc: xen-devel@lists.xensource.com, xen-users@lists.xensource.com > > > Date: Monday, March 1, 2010, 2:52 PM > > > > > > Why using grub2 vs grub is baffling am not sure.... > > > > > > I think I'd prefer to use the kernel command line in grub2 so that the > options may be changed easier laster on. However I still have been having > so success with boot > > > XEN with Ubuntu's grub2 boot loader. > > > > > > Am I correct to conclude CONFIG_CMDLINE_BOL must = N to allow GRUB2 to > pass kernel options? > > > [1]http://cateee.net/lkddb/web-lkddb/CMDLINE_BOOL.html > > > > > > I have read through the Wiki page at > [2]http://wiki.xensource.com/xenwiki/XenCommonProblems. I am running a > 64-bit Xen Hypervisor. > > > > > > Thus far I have been having no success with using the example entry > found at > [3]http://old.nabble.com/Strange-interaction-from-grub2-and-XEN-td26464067.html > > > > > > A question regarding that entry is what would be the root device when > there are separate /boot and root partitions? I am using the partition > with the /boot (grub2) as > > > my 'set root' device right? > > > > > > I am also using the dummy=dummy as the first option on the first > module line, again as discovered on the wiki. > > > > > > Additionally posted on another thread here in the Xen mailing list > was: > > > > > > > On 1 March 2010 08:09, Luke Carrier wrote: > > > >> > > > >> Not sure about Debian, but I know Ubuntu isn*t ideal for Xen any > more > > > >> because of the presence of GRUB2 * the new boot loader doesn*t > support > > > >> passing certain command lines and the current multiboot > implementation > > > >> breaks the method Xen uses to load the > > > Dom0 kernel * you can*t use Ubuntu > > > >> 9.10 Server with Xen. > > > > > > I am confused now because Boris clearly created a wordpress.com weblog > in where Ubuntu 9.10 Server was > > > used with XEN. > > > > > > Lastly, out of curiosity, what (or what kind) of commands cannot be > passed with GRUB2? > > > > > > ThX! > > > > > > > > > -M > > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > Date: Sat, 27 Feb 2010 11:20:31 -0800 > > > From: bderzhavets@yahoo.com > > > Subject: Re: [Xen-devel] RE: [Xen-users] How to: Improve Network > Throughput of a XEN kernel? > > > To: mike.viau@sheridanc.on.ca > > > CC: xen-devel@lists.xensource.com; xen-users@lists.xensource.com > > > > > > In Xen Wiki is mentioned , that parameters for kernel may be set via > command line as follows:- > > > > ************************************************************************************************************ > > > module (hd1,10)/vmlinuz-2.6.31.8 dummy=dummy root=/dev/sda7 ro > console=tty0 > > > > ************************************************************************************************************ > > > # cat /proc/cmdline > > > > > > shows that GRUB2 skips first argument, what causes problem. > > > > > > Compiling built-in CMD_LINE is just an option. > > > Read all Wiki written by Pasi and you would never be surprised ;) > > > > > > > P.S: What is your feeling on using a recent Ubuntu system with > > > > grub-legacy as the boot loader? I noticed that Squeeze has adopted > grub2 > package as well too... > > > > > > I have development Xen 3.4.3 instance on top Ubuntu 9.10 Server with > GRUB2 ( baremetall) loading any from 3 mentioned bellow kernels under Xen > 3.4.3 > > > > > > 1.) 2.6.32.9 PVOPS > > > 2.) 2.6.31.6 PVOPS > > > 3.) 2.6.31.12 xenified > > > > > > with no problems with dummy=dummy as first parameter ( thanks Pasi). > > > It may also may be done via compiling kernels with built-in CMD_LINE. > > > It works fine as well. > > > > > > Boris. > > > > > > --- On Sat, 2/27/10, Mike Viau wrote: > > > > > > From: Mike Viau > > > Subject: [Xen-devel] RE: [Xen-users] How to: Improve Network > Throughput of a XEN kernel? > > > To: bderzhavets@yahoo.com > > > Cc: xen-devel@lists.xensource.com, xen-users@lists.xensource.com > > > Date: Saturday, February 27, 2010, 12:03 PM > > > > > > Yes, thank you Boris for pointing that out. I found that was defiantly > true. The 3.2.1 xen hypervisor could only boot the 2.6.26-2 xen kernel and > the xenified > > > 2.6.31.12. > > > > > > Additionally I found that: > > > > > > Xend does not start when using pv_ops dom0 kernel? > > > > > > In December 2009 pv_ops dom0 kernel modules were renamed to have a > "xen-" prefix in them, ie. "evtchn.ko" became "xen-evtchn.ko". > > > > > > This makes Xen 3.4.x xend fail to start, because it tried to load > "evtchn.ko", but that doesn't exist. You need to load "xen-evtchn.ko" and > then start xend. Fedora 12 > > > xen-3.4.2-2 rpms have this problem fixed. > > > > > > Also make sure you have xenfs mounted to "/proc/xen", that's needed > aswell. > > > > > > [Source: [4]http://wiki.xensource.com/xenwiki/XenParavirtOps] > > > > > > And it appears the Debian xen-3.4.2-2 debs (in squeeze) have not fixed > this problem. > > > > > > Lastly I am still doing some reading on the grub2 package and how to > configure it. It turns out that I have been using grub-legacy (or just > good old grub) for so long > > > that configuring the grub2 boot loader is turning out to be my own > personal kryptonite. > > > > > > > > > > > > I am still working on getting the hypervisor to load the dom0 kernel > in a Ubuntu system. > > > > > > > > > > > > I have noticed an example in your wordpress @ [5]Set up Xen 3.4.3-rc2 > & Libvirt 0.7.0 Dom0 (with 2.6.31.8 xenified aka Suse kernel) on top of > Ubuntu 9.10 Server > > > > > > > > > > > > With CONFIG_CMDLINE="root=/dev/sdb11 ro console=tty0" > > > > > > > > > > > > menuentry "Xen 3.4 / Ubuntu 9.10 kernel 2.6.31.8 xenified" { > > > insmod ext2 > > > set root=(hd1,10) > > > multiboot (hd1,10)/xen-3.4.gz > > > module (hd1,10)/vmlinuz-2.6.31.8 > > > module (hd1,10)/initrd-2.6.31.8.img > > > } > > > > > > > > > > > > I am sure you wouldn't of mentioned the CONFIG_CMDLINE kernel option > is it was not important for using with grub2 right? I have not have to > compile a kernel with the > > > command line previous as I was just able to supply the arguments on > the kernel line of my menu.lst (grub configuration file). > > > > > > P.S: What is your feeling on using a recent Ubuntu system with > grub-legacy as the boot loader? I noticed that Squeeze has adopted grub2 > package as well too... > > > > > > > > > -M > > > > > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > > > > > > Date: Sat, 27 Feb 2010 04:32:59 -0800 > > > From: bderzhavets@yahoo.com > > > Subject: Re: FW: [Xen-users] How to: Improve Network Throughput of a > XEN kernel? > > > To: jbeulich@novell.com; mike.viau@sheridanc.on.ca > > > CC: xen-devel@lists.xensource.com; xen-users@lists.xensource.com > > > > > > You cannot load pvops kernel under xen-hypervisor 3.2.1 ( >=3.4.0 in > general) > > > > > > Boris. > > > > > > --- On Fri, 2/26/10, Mike Viau wrote: > > > > > > From: Mike Viau > > > Subject: FW: [Xen-users] How to: Improve Network Throughput of a XEN > kernel? > > > To: bderzhavets@yahoo.com, jbeulich@novell.com > > > Cc: xen-users@lists.xensource.com, xen-devel@lists.xensource.com > > > Date: Friday, February 26, 2010, 3:19 PM > > > > > > Hello again, > > > > > > Just a quick update... > > > > > > I was really hoping it was just going to be an issue with the kernel > .config, however I copied exact configuration from the working Debain > 2.6.26-2-xen-amd64 kernel > > > and used a make oldconfig but the end result was the same. > > > > > > After booting the xenified (2.6.31.12) or pvops kernel (2.6.31.6) via > both xen-hypervisor 3.2.1 or xen-hypervisor 3.4.2 the maximum data > transfer to/from the Dom0 > > > (Xen host) is ~1MB/s. > > > > > > Next I plan on re-trying my efforts on a Ubuntu system to see if same > network bottlenecking is present. > > > > > > > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > Date: Fri, 26 Feb 2010 09:16:59 -0800 > > > From: bderzhavets@yahoo.com > > > Subject: Re: [Xen-users] How to: Improve Network Throughput of a XEN > kernel? > > > To: xen-users@lists.xensource.com; viaum@sheridanc.on.ca > > > > > > Would escalate to xen-devel. 2.6.31.8 (12) aka Suse xenified was Jan > Beulich stuff. > > > He might have an answer. > > > Boris. > > > > > > --- On Fri, 2/26/10, Mike Viau wrote: > > > > > > From: Mike Viau > > > Subject: [Xen-users] How to: Improve Network Throughput of a XEN > kernel? > > > To: xen-users@lists.xensource.com > > > Cc: waldi@debian.org > > > Date: Friday, February 26, 2010, 12:09 PM > > > > > > Hi there, > > > > > > I am starting a new thread to try and solve a problem I am having will > slow network throughput to the XEN host (not to a domU). > > > > > > I have confirmed my problem occurs with PVOPS 'xen/master' kernel as > well using > > > > [6]http://pasik.reaktio.net/xen/pv_ops-dom0-debug/config-2.6.31.6-pvops-dom0-xen-master-x86_64 > as the .config. > > > > > > It also occurs with a forward ported xenified kernel (2.6.31.12) using > the attached config-11-xen (.config). > > > > > > However when using the xen kernel found in debain lenny (2.6.26-2) for > the amd64 architecture I get similiar transfer rates to my Xen host as I > do on a baremetel > > > kernel. > > > > > > Bastian Blank perhaps you could point me in the right direction in > regards to a debian specific customization or patch I should be aware of? > > > Maxium transfer rate (to/from hard disk) using the PVOPS 'xen/master' > kernel and forward ported xenified kernel (2.6.31.12) is bottlenecking at > ~1MB/s. On the > > > debain lenny xen kernel I am getting ~4.5MB/s. > > > > > > I was hoping one could suggest options in the .config that might be > modified in order to improve network throughput of a XEN kernel and > therefor increase the data > > > transfer rates over the netowork to and from the Xen host. > > > > > > Thanks alot for any help! As always I will be willing to accept any > suggestions :) > > > > > > -M > > > > > > > > -------------------------------------------------------------------------- > > Live connected with Messenger on your phone [1]Learn more. > > References > > Visible links > 1. http://go.microsoft.com/?linkid=9712957