xen affinity in an SMP system

xen affinity in an SMP system

[James Harper]

As I understand it, a domain is tied to a specific CPU on startup, and
if it can be changed, it can only be changed manually. Is that right?

Is it possible to have xen (or xend) check at regular intervals and move
a domain to another physical CPU depending on current load?

James


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl

Re: xen affinity in an SMP system

[Mark A. Williamson]

> As I understand it, a domain is tied to a specific CPU on startup, and
> if it can be changed, it can only be changed manually. Is that right?

Yup, it's changed manually using an xm command (probably "xm pincpu [dom] 
[cpu]" but I've never used it and could be wrong ;-).

> Is it possible to have xen (or xend) check at regular intervals and move
> a domain to another physical CPU depending on current load?

Currently this isn't done but it's perfectly feasible.  Sensible options are:
1) Have an scheduler in Xen that will migrate domains about at run time.
2) Have the load balancer task in Xend.

2 is possibly the easiest to do and arguably the most flexible since it makes 
it easy to provide pluggable balancing policy modules.  1 would make sure 
that CPUs remain idle for the minimum time possible.

It's possible that one (or both) of these solutions will get implemented at 
some stage, although (I suspect) not in time for 2.0 which is looking pretty 
imminent.  I don't know of anybody local who's volunteered to do this, 
however.

Cheers,
Mark

>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
> Use IT products in your business? Tell us what you think of them. Give us
> Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
> http://productguide.itmanagersjournal.com/guidepromo.tmpl
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl

Re: xen affinity in an SMP system

[Keir Fraser]

> > As I understand it, a domain is tied to a specific CPU on startup, and
> > if it can be changed, it can only be changed manually. Is that right?
> 
> Yup, it's changed manually using an xm command (probably "xm pincpu [dom] 
> [cpu]" but I've never used it and could be wrong ;-).

Yes, that works.

> > Is it possible to have xen (or xend) check at regular intervals and move
> > a domain to another physical CPU depending on current load?
> 
> Currently this isn't done but it's perfectly feasible.  Sensible options are:
> 1) Have an scheduler in Xen that will migrate domains about at run time.
> 2) Have the load balancer task in Xend.

Alternatively you could do it with a small external program. This
would poll either xend to get load stats for each domain, or could use
libxc directly. It would then talk to xend to re-pin appropriate
domains.

A nice simple modular piece of functionality that would be easy to
maintain and modify.

 -- Keir


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl

Problems with loopback and sda1

[Rune Johan Andresen]

[-- Attachment #1: Type: text/plain, Size: 757 bytes --]

Hi, my pysical HD is mounted on the logical sda1. When i try to mount
the loopback on something else than sda1, xm complains.

With:
disk = [ 'phy:loop0,xda1,w', 'phy:sda1,sda1,r' ]
root = "/dev/xda1 ro"

I get error msg:
 >>error begin
VFS: Cannot open root device "xda1" or unknown-block(2,0)
   Please append a correct "root=" boot option
   Kernel panic: VFS: Unable to mount root fs on unknown-block(2,0)
 >>error end


Does Xen deny use of anything else than logical sda1 for HD? When my
physical HD is sda1, how can I use another logical drive for my 
loopback?

To save space I want to read-only mount my pysical HD from the 
loopbacks.
(If I try to mount sda1 now, I mount the loopback, not the pysical HD)

Cheers,
Rune


[-- Attachment #2: Type: text/enriched, Size: 1205 bytes --]

Hi, my pysical HD is mounted on the logical sda1. When i try to mount 

the loopback on something else than sda1, xm complains. 


With:

<fontfamily><param>Geneva</param><bigger>disk = [ 'phy:loop0,xda1,w',
'phy:sda1,sda1,r' ]

root = "/dev/xda1 ro"


</bigger></fontfamily>I get error msg:

>>error begin

<fontfamily><param>Geneva</param><bigger>VFS: Cannot open root device
"xda1" or unknown-block(2,0)

</bigger></fontfamily><fontfamily><param>Lucida Grande</param><bigger>   </bigger></fontfamily><fontfamily><param>Geneva</param><bigger>Please
append a correct "root=" boot option              

</bigger></fontfamily><fontfamily><param>Lucida Grande</param><bigger>   </bigger></fontfamily><fontfamily><param>Geneva</param><bigger>Kernel
panic: VFS: Unable to mount root fs on unknown-block(2,0)

>>error end

</bigger></fontfamily>


Does Xen deny use of anything else than logical sda1 for HD? When my 

physical HD is sda1, how can I use another logical drive for my
loopback? 


To save space I want to read-only mount my pysical HD from the
loopbacks.

(If I try to mount sda1 now, I mount the loopback, not the pysical HD) 


Cheers,

Rune 

Re: Problems with loopback and sda1

[Ian Pratt]

> Hi, my pysical HD is mounted on the logical sda1. When i try to mount
> the loopback on something else than sda1, xm complains.
> 
> With:
> disk = [ 'phy:loop0,xda1,w', 'phy:sda1,sda1,r' ]
> root = "/dev/xda1 ro"

Why not just use sda2, sdb1, or hda1?

I think we did away with xda because no-one was using it. Should
we should put it back?

Ian


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl

Re: Problems with loopback and sda1

[Mark A. Williamson]

> With:
> disk = [ 'phy:loop0,xda1,w', 'phy:sda1,sda1,r' ]
> root = "/dev/xda1 ro"
>
> I get error msg:
>  >>error begin
>
> VFS: Cannot open root device "xda1" or unknown-block(2,0)
>    Please append a correct "root=" boot option
>    Kernel panic: VFS: Unable to mount root fs on unknown-block(2,0)

The xda devices got removed.  Any sd* or hd* device node should work instead 
though.

Did you realise you can now specify file:/path/to/file as the source device 
instead of running losetup manually and using phy:loop0?

HTH,
Mark

>  >>error end
>
> Does Xen deny use of anything else than logical sda1 for HD? When my
> physical HD is sda1, how can I use another logical drive for my
> loopback?
>
> To save space I want to read-only mount my pysical HD from the
> loopbacks.
> (If I try to mount sda1 now, I mount the loopback, not the pysical HD)
>
> Cheers,
> Rune


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl

Re: Problems with loopback and sda1

[Rune Johan Andresen]

The sda2 worked out fine! I did not know that you can specify the file
directly. This is good news tough:-)

Thank you!

Cheers,
Rune


On Oct 11, 2004, at 4:51 PM, Mark A. Williamson wrote:

>> With:
>> disk = [ 'phy:loop0,xda1,w', 'phy:sda1,sda1,r' ]
>> root = "/dev/xda1 ro"
>>
>> I get error msg:
>>>> error begin
>>
>> VFS: Cannot open root device "xda1" or unknown-block(2,0)
>>    Please append a correct "root=" boot option
>>    Kernel panic: VFS: Unable to mount root fs on unknown-block(2,0)
>
> The xda devices got removed.  Any sd* or hd* device node should work 
> instead
> though.
>
> Did you realise you can now specify file:/path/to/file as the source 
> device
> instead of running losetup manually and using phy:loop0?
>
> HTH,
> Mark
>
>>>> error end
>>
>> Does Xen deny use of anything else than logical sda1 for HD? When my
>> physical HD is sda1, how can I use another logical drive for my
>> loopback?
>>
>> To save space I want to read-only mount my pysical HD from the
>> loopbacks.
>> (If I try to mount sda1 now, I mount the loopback, not the pysical HD)
>>
>> Cheers,
>> Rune
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: IT Product Guide on 
> ITManagersJournal
> Use IT products in your business? Tell us what you think of them. Give 
> us
> Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out 
> more
> http://productguide.itmanagersjournal.com/guidepromo.tmpl
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel



-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl

Re: xen affinity in an SMP system

[Paul Dorman]

On Monday 11 October 2004 10:01 pm, Keir Fraser wrote:
> > > As I understand it, a domain is tied to a specific CPU on startup, and
> > > if it can be changed, it can only be changed manually. Is that right?
> >
> > Yup, it's changed manually using an xm command (probably "xm pincpu [dom]
> > [cpu]" but I've never used it and could be wrong ;-).
>
> Yes, that works.
>
> > > Is it possible to have xen (or xend) check at regular intervals and
> > > move a domain to another physical CPU depending on current load?
> >
> > Currently this isn't done but it's perfectly feasible.  Sensible options
> > are: 1) Have an scheduler in Xen that will migrate domains about at run
> > time. 2) Have the load balancer task in Xend.
>
> Alternatively you could do it with a small external program. This
> would poll either xend to get load stats for each domain, or could use
> libxc directly. It would then talk to xend to re-pin appropriate
> domains.
>
> A nice simple modular piece of functionality that would be easy to
> maintain and modify.

This is the approach I like for sure. It's much easier for someone like me 
(who isn't a low-level systems programmer) to write scripts to do this sort 
of stuff. I think you'll get much more community input this way too, and it 
doesn't negate the possibility of building load-balancing capabilities in at 
a later date.

I intend to use a combination of tools on the VMs themselves, reporting back 
to a central control script, and whatever auxilliary information I can get 
from Xen and dom-0's proc.

Paul


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl