Xen default install - no cpufreq on dom0 ?

Xen default install - no cpufreq on dom0 ?

[Luis R. Rodriguez]

For a while now I've been installing Xen from source on my Debian box,
with a default configuration always. This is party why I've run into
compile issues before it seems, but I've also just picked up on
another possible issue. My system on dom0 overheats and shuts itself
off. For a while I didn't think this was Xen related and perhaps
related to the fact that I built this system myself. However, after
running Debian without Xen (rm -rf /boot/xen*; update-grub; reboot;
systemd-detect-virt yield none) it would seem this does not happen.
What's a bit concerning though is even if I run offlineimap in
singlethreaded mode (offlineimap -1) the machine overheats and shuts
itself off, so I don't necessarily need to be doing a lot of work. I
tried compiling the kernel without Xen while running offlineimap as
well with make -j 8 and it doesn't overheat, meanwhile on Xen dom0
just using offlineimap -1 seems to trigger a shutoff rather quickly.

The kernel has:

CONFIG_CPU_FREQ_GOV_PERFORMANCE=y

I tried on /etc/default/grub

GRUB_CMDLINE_LINUX_XEN_REPLACE="cpufreq=dom0-kernel"

But that didn't help. This is running 4.5.0-rc2 and the latest Xen I
was able to build (I try xen staging tree always).

  Luis

Re: Xen default install - no cpufreq on dom0 ?

[Konrad Rzeszutek Wilk]

On Tue, Feb 23, 2016 at 12:48:18PM -0800, Luis R. Rodriguez wrote:
> For a while now I've been installing Xen from source on my Debian box,
> with a default configuration always. This is party why I've run into
> compile issues before it seems, but I've also just picked up on
> another possible issue. My system on dom0 overheats and shuts itself
> off. For a while I didn't think this was Xen related and perhaps
> related to the fact that I built this system myself. However, after
> running Debian without Xen (rm -rf /boot/xen*; update-grub; reboot;
> systemd-detect-virt yield none) it would seem this does not happen.
> What's a bit concerning though is even if I run offlineimap in
> singlethreaded mode (offlineimap -1) the machine overheats and shuts
> itself off, so I don't necessarily need to be doing a lot of work. I
> tried compiling the kernel without Xen while running offlineimap as
> well with make -j 8 and it doesn't overheat, meanwhile on Xen dom0
> just using offlineimap -1 seems to trigger a shutoff rather quickly.

What exactly triggers the shutoff.
> 
> The kernel has:
> 
> CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
> 
> I tried on /etc/default/grub
> 
> GRUB_CMDLINE_LINUX_XEN_REPLACE="cpufreq=dom0-kernel"
> 
> But that didn't help. This is running 4.5.0-rc2 and the latest Xen I
> was able to build (I try xen staging tree always).

Right.  We are actually going to remove that option as it makes no sense
for Linux (dom0) to control the cpu frequency.

Irrespective of that - there are simply no drivers in dom0 that would
do the cpufreq for you (unless you recompile the kernel and edit in
arch/x86/xen/enlighten.c the calls to disable cpuidle and cpufreq code).

> 
>   Luis
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Xen default install - no cpufreq on dom0 ?

[Luis R. Rodriguez]

On Tue, Feb 23, 2016 at 05:00:12PM -0500, Konrad Rzeszutek Wilk wrote:
> On Tue, Feb 23, 2016 at 12:48:18PM -0800, Luis R. Rodriguez wrote:
> > For a while now I've been installing Xen from source on my Debian box,
> > with a default configuration always. This is party why I've run into
> > compile issues before it seems, but I've also just picked up on
> > another possible issue. My system on dom0 overheats and shuts itself
> > off. For a while I didn't think this was Xen related and perhaps
> > related to the fact that I built this system myself. However, after
> > running Debian without Xen (rm -rf /boot/xen*; update-grub; reboot;
> > systemd-detect-virt yield none) it would seem this does not happen.
> > What's a bit concerning though is even if I run offlineimap in
> > singlethreaded mode (offlineimap -1) the machine overheats and shuts
> > itself off, so I don't necessarily need to be doing a lot of work. I
> > tried compiling the kernel without Xen while running offlineimap as
> > well with make -j 8 and it doesn't overheat, meanwhile on Xen dom0
> > just using offlineimap -1 seems to trigger a shutoff rather quickly.
> 
> What exactly triggers the shutoff.

Clearly, overheating. I'm not yet sure if its just BIOS/kernel/hypervisor
induced forced shutdown. Will try again, but this is rather intrusive as
its on my personal home system.

> > The kernel has:
> > 
> > CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
> > 
> > I tried on /etc/default/grub
> > 
> > GRUB_CMDLINE_LINUX_XEN_REPLACE="cpufreq=dom0-kernel"
> > 
> > But that didn't help. This is running 4.5.0-rc2 and the latest Xen I
> > was able to build (I try xen staging tree always).
> 
> Right.  We are actually going to remove that option as it makes no sense
> for Linux (dom0) to control the cpu frequency.
> 
> Irrespective of that - there are simply no drivers in dom0 that would
> do the cpufreq for you (unless you recompile the kernel and edit in
> arch/x86/xen/enlighten.c the calls to disable cpuidle and cpufreq code).

How can I (and Xen by default) not overheat a system then ?
What's the short term and long term plan? I was a bit surprised
a simple single threaded offlineimap would force a shutoff on
Xen. I figured I must be seriously misconfiguring something
but this is just running with defaults.

  Luis

Re: Xen default install - no cpufreq on dom0 ?

[Konrad Rzeszutek Wilk]

On Tue, Feb 23, 2016 at 11:18:09PM +0100, Luis R. Rodriguez wrote:
> On Tue, Feb 23, 2016 at 05:00:12PM -0500, Konrad Rzeszutek Wilk wrote:
> > On Tue, Feb 23, 2016 at 12:48:18PM -0800, Luis R. Rodriguez wrote:
> > > For a while now I've been installing Xen from source on my Debian box,
> > > with a default configuration always. This is party why I've run into
> > > compile issues before it seems, but I've also just picked up on
> > > another possible issue. My system on dom0 overheats and shuts itself
> > > off. For a while I didn't think this was Xen related and perhaps
> > > related to the fact that I built this system myself. However, after
> > > running Debian without Xen (rm -rf /boot/xen*; update-grub; reboot;
> > > systemd-detect-virt yield none) it would seem this does not happen.
> > > What's a bit concerning though is even if I run offlineimap in
> > > singlethreaded mode (offlineimap -1) the machine overheats and shuts
> > > itself off, so I don't necessarily need to be doing a lot of work. I
> > > tried compiling the kernel without Xen while running offlineimap as
> > > well with make -j 8 and it doesn't overheat, meanwhile on Xen dom0
> > > just using offlineimap -1 seems to trigger a shutoff rather quickly.
> > 
> > What exactly triggers the shutoff.
> 
> Clearly, overheating. I'm not yet sure if its just BIOS/kernel/hypervisor
> induced forced shutdown. Will try again, but this is rather intrusive as
> its on my personal home system.

OK, serial console output pls with full debug :-)
> 
> > > The kernel has:
> > > 
> > > CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
> > > 
> > > I tried on /etc/default/grub
> > > 
> > > GRUB_CMDLINE_LINUX_XEN_REPLACE="cpufreq=dom0-kernel"
> > > 
> > > But that didn't help. This is running 4.5.0-rc2 and the latest Xen I
> > > was able to build (I try xen staging tree always).
> > 
> > Right.  We are actually going to remove that option as it makes no sense
> > for Linux (dom0) to control the cpu frequency.
> > 
> > Irrespective of that - there are simply no drivers in dom0 that would
> > do the cpufreq for you (unless you recompile the kernel and edit in
> > arch/x86/xen/enlighten.c the calls to disable cpuidle and cpufreq code).
> 
> How can I (and Xen by default) not overheat a system then ?
> What's the short term and long term plan? I was a bit surprised
> a simple single threaded offlineimap would force a shutoff on
> Xen. I figured I must be seriously misconfiguring something
> but this is just running with defaults.

No idea as I never had this issue on my systems. And I've a ton of different
hardware (desktops, laptops, servers, etc) running with Xen.

> 
>   Luis

Re: Xen default install - no cpufreq on dom0 ?

[Andrew Cooper]

On 23/02/2016 22:18, Luis R. Rodriguez wrote:
> On Tue, Feb 23, 2016 at 05:00:12PM -0500, Konrad Rzeszutek Wilk wrote:
>> On Tue, Feb 23, 2016 at 12:48:18PM -0800, Luis R. Rodriguez wrote:
>>> For a while now I've been installing Xen from source on my Debian box,
>>> with a default configuration always. This is party why I've run into
>>> compile issues before it seems, but I've also just picked up on
>>> another possible issue. My system on dom0 overheats and shuts itself
>>> off. For a while I didn't think this was Xen related and perhaps
>>> related to the fact that I built this system myself. However, after
>>> running Debian without Xen (rm -rf /boot/xen*; update-grub; reboot;
>>> systemd-detect-virt yield none) it would seem this does not happen.
>>> What's a bit concerning though is even if I run offlineimap in
>>> singlethreaded mode (offlineimap -1) the machine overheats and shuts
>>> itself off, so I don't necessarily need to be doing a lot of work. I
>>> tried compiling the kernel without Xen while running offlineimap as
>>> well with make -j 8 and it doesn't overheat, meanwhile on Xen dom0
>>> just using offlineimap -1 seems to trigger a shutoff rather quickly.
>> What exactly triggers the shutoff.
> Clearly, overheating. I'm not yet sure if its just BIOS/kernel/hypervisor
> induced forced shutdown. Will try again, but this is rather intrusive as
> its on my personal home system.

You have dislodged the heat sync and the processor is shutting down due
to thermal cutout.

This is a hardware issue, not a software issue.

~Andrew

Re: Xen default install - no cpufreq on dom0 ?

[Luis R. Rodriguez]

On Tue, Feb 23, 2016 at 10:44:56PM +0000, Andrew Cooper wrote:
> On 23/02/2016 22:18, Luis R. Rodriguez wrote:
> > On Tue, Feb 23, 2016 at 05:00:12PM -0500, Konrad Rzeszutek Wilk wrote:
> >> On Tue, Feb 23, 2016 at 12:48:18PM -0800, Luis R. Rodriguez wrote:
> >>> For a while now I've been installing Xen from source on my Debian box,
> >>> with a default configuration always. This is party why I've run into
> >>> compile issues before it seems, but I've also just picked up on
> >>> another possible issue. My system on dom0 overheats and shuts itself
> >>> off. For a while I didn't think this was Xen related and perhaps
> >>> related to the fact that I built this system myself. However, after
> >>> running Debian without Xen (rm -rf /boot/xen*; update-grub; reboot;
> >>> systemd-detect-virt yield none) it would seem this does not happen.
> >>> What's a bit concerning though is even if I run offlineimap in
> >>> singlethreaded mode (offlineimap -1) the machine overheats and shuts
> >>> itself off, so I don't necessarily need to be doing a lot of work. I
> >>> tried compiling the kernel without Xen while running offlineimap as
> >>> well with make -j 8 and it doesn't overheat, meanwhile on Xen dom0
> >>> just using offlineimap -1 seems to trigger a shutoff rather quickly.
> >> What exactly triggers the shutoff.
> > Clearly, overheating. I'm not yet sure if its just BIOS/kernel/hypervisor
> > induced forced shutdown. Will try again, but this is rather intrusive as
> > its on my personal home system.
> 
> You have dislodged the heat sync and the processor is shutting down due
> to thermal cutout.

How do I prevent this ? The difference in operation between dom0 and
bare metal is rather extreme. Is this expected ? Is there a clutch
I can use ?

  Luis

Re: Xen default install - no cpufreq on dom0 ?

[Andrew Cooper]

On 23/02/2016 23:44, Luis R. Rodriguez wrote:
> On Tue, Feb 23, 2016 at 10:44:56PM +0000, Andrew Cooper wrote:
>> On 23/02/2016 22:18, Luis R. Rodriguez wrote:
>>> On Tue, Feb 23, 2016 at 05:00:12PM -0500, Konrad Rzeszutek Wilk wrote:
>>>> On Tue, Feb 23, 2016 at 12:48:18PM -0800, Luis R. Rodriguez wrote:
>>>>> For a while now I've been installing Xen from source on my Debian box,
>>>>> with a default configuration always. This is party why I've run into
>>>>> compile issues before it seems, but I've also just picked up on
>>>>> another possible issue. My system on dom0 overheats and shuts itself
>>>>> off. For a while I didn't think this was Xen related and perhaps
>>>>> related to the fact that I built this system myself. However, after
>>>>> running Debian without Xen (rm -rf /boot/xen*; update-grub; reboot;
>>>>> systemd-detect-virt yield none) it would seem this does not happen.
>>>>> What's a bit concerning though is even if I run offlineimap in
>>>>> singlethreaded mode (offlineimap -1) the machine overheats and shuts
>>>>> itself off, so I don't necessarily need to be doing a lot of work. I
>>>>> tried compiling the kernel without Xen while running offlineimap as
>>>>> well with make -j 8 and it doesn't overheat, meanwhile on Xen dom0
>>>>> just using offlineimap -1 seems to trigger a shutoff rather quickly.
>>>> What exactly triggers the shutoff.
>>> Clearly, overheating. I'm not yet sure if its just BIOS/kernel/hypervisor
>>> induced forced shutdown. Will try again, but this is rather intrusive as
>>> its on my personal home system.
>> You have dislodged the heat sync and the processor is shutting down due
>> to thermal cutout.
> How do I prevent this ?

Open up your computer and shuffle things around.  Alternatively unclog
the fan.  Possibly both.

> The difference in operation between dom0 and
> bare metal is rather extreme. Is this expected ? Is there a clutch
> I can use ?

The only thing software can control is now badly things go wrong.  It is
not the source of the problem.

Baremetal probably has deep C and P states set up properly, but it will
still be under clocking as much as it can in an attempt to avoid
generating as much heat.

~Andrew

Re: Xen default install - no cpufreq on dom0 ?

[Luis R. Rodriguez]

On Tue, Feb 23, 2016 at 11:55:48PM +0000, Andrew Cooper wrote:
> On 23/02/2016 23:44, Luis R. Rodriguez wrote:
> > On Tue, Feb 23, 2016 at 10:44:56PM +0000, Andrew Cooper wrote:
> >> On 23/02/2016 22:18, Luis R. Rodriguez wrote:
> >>> On Tue, Feb 23, 2016 at 05:00:12PM -0500, Konrad Rzeszutek Wilk wrote:
> >>>> On Tue, Feb 23, 2016 at 12:48:18PM -0800, Luis R. Rodriguez wrote:
> >>>>> For a while now I've been installing Xen from source on my Debian box,
> >>>>> with a default configuration always. This is party why I've run into
> >>>>> compile issues before it seems, but I've also just picked up on
> >>>>> another possible issue. My system on dom0 overheats and shuts itself
> >>>>> off. For a while I didn't think this was Xen related and perhaps
> >>>>> related to the fact that I built this system myself. However, after
> >>>>> running Debian without Xen (rm -rf /boot/xen*; update-grub; reboot;
> >>>>> systemd-detect-virt yield none) it would seem this does not happen.
> >>>>> What's a bit concerning though is even if I run offlineimap in
> >>>>> singlethreaded mode (offlineimap -1) the machine overheats and shuts
> >>>>> itself off, so I don't necessarily need to be doing a lot of work. I
> >>>>> tried compiling the kernel without Xen while running offlineimap as
> >>>>> well with make -j 8 and it doesn't overheat, meanwhile on Xen dom0
> >>>>> just using offlineimap -1 seems to trigger a shutoff rather quickly.
> >>>> What exactly triggers the shutoff.
> >>> Clearly, overheating. I'm not yet sure if its just BIOS/kernel/hypervisor
> >>> induced forced shutdown. Will try again, but this is rather intrusive as
> >>> its on my personal home system.
> >> You have dislodged the heat sync and the processor is shutting down due
> >> to thermal cutout.
> > How do I prevent this ?
> 
> Open up your computer and shuffle things around.  Alternatively unclog
> the fan.  Possibly both.

This wasn't it. It would seem some AMD systems have a boost option
available, disabling that fixes my woes:

echo 0 > /sys/devices/system/cpu/cpufreq/boost

With this my Linux box without Xen survices the most extreme of situations.

On Linux this is implemented as a cpu freq driver callback
cpufreq_driver->set_boost(state), see cpufreq_boost_trigger_state()., on
drivers/cpufreq/cpufreq.c.

I don't find any of this on the Xen hypervisor. How can I address this
on Xen?

> > The difference in operation between dom0 and
> > bare metal is rather extreme. Is this expected ? Is there a clutch
> > I can use ?
> 
> The only thing software can control is now badly things go wrong.  It is
> not the source of the problem.
> 
> Baremetal probably has deep C and P states set up properly, but it will
> still be under clocking as much as it can in an attempt to avoid
> generating as much heat.

Should be under clocking..

  Luis

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel