On Thu, Sep 3, 2015 at 12:06 PM, Jintack Lim <jintack@cs.columbia.edu> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">

Hi,

I was= trying to set irq affinity by writing a value to=C2=A0/proc/irq/<irq>= ;/smp_affinity,

but smp_affinity value was not changed at all.

Ian suggested to take this to the devel list.

I'm wor= king on Xen4.5, ARMv8.

I checke= d that setting smp_affinity works on Xen4.5, x86.

So, probably th= is is an ARM specific issue.

=C2=A0

Thanks,
Jintack
=

On Thu, Sep 3, 2015 at 11:34 AM, Ian Campbell <<= a href=3D"mailto:ian.campbell@citrix.com" target=3D"_blank" onclick=3D"wind= ow.open('https://mail.google.com/mail/?view=3Dcm&tf=3D1&to=3Dia= n.campbell@citrix.com&cc=3D&bcc=3D&su=3D&body=3D','= _blank');return false;">ian.campbell@citrix.com> wrote:
On Thu, 2015-09-03 at 10:59 -0400, Jintack Lim= wrote:
> Hi,
>
> While I was running Apache server,
> I found that one of Dom0 vcpu is running 100% to handle irqs,
> and those irqs are set to be processed only on that specific vcpu.
>
> Referring to this document,
> http://wiki.xen.org/wiki/Netwo= rk_Throughput_and_Performance_Guide
> I tried to change smp_affinity by writing a value to /proc/irq/<irq=
> -no>/smp_affinity,
> however the smp_affinity value was not changed.
>
> I'm working on Xen 4.5 on ARMv8,
> and the irq is netback tx interrupt.
>
> # cat /proc/irq/106/smp_affinity
> 1
>
> # cat /proc/interrupts
> ...
> 106:=C2=A0 =C2=A0 =C2=A0 53849=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0= =C2=A0xen-dyn-event
> vif1.1-q0-tx
> 107:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 0=C2=A0 =C2=A0xen-dyn-event
> vif1.1-q0-rx
> 108:=C2=A0 =C2=A0 =C2=A0 61460=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0= =C2=A0xen-dyn-event
> vif1.1-q1-tx
> 109:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 0=C2=A0 =C2=A0xen-dyn-event
> vif1.1-q1-rx
> 110:=C2=A0 =C2=A0 =C2=A0 67118=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0= =C2=A0xen-dyn-event
> vif1.1-q2-tx
> 111:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 0=C2=A0 =C2=A0xen-dyn-event
> vif1.1-q2-rx
> 112:=C2=A0 =C2=A0 =C2=A0 58273=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0= =C2=A0xen-dyn-event
> vif1.1-q3-tx
> 113:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 0=C2=A0 =C2=A0xen-dyn-event
> vif1.1-q3-rx
> ...
>
> What would be the way to change smp_affinity?

It should be via /proc/irq/<irq>/smp_affinity as you'= ve tried, I asked
Stefano and he things this was already implemented in 4.5 even. That it is<= br> not happening would be a bug, I think.

I'd suggest you take this to the devel list as a bug.

> and where is the affinity set initially for netback tx interrupts?

Not, sure, I think it might just be generic IRQ code.

Ian.

--047d7bdc14e638651c051edb2216-- --===============0315581464034028361== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============0315581464034028361==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Julien Grall Subject: Re: [Xen-users] Changing netback tx interrupts affinity on Dom0 Date: Thu, 3 Sep 2015 19:21:43 +0100 Message-ID: <55E88FB7.6020004@citrix.com> References: <1441294496.26292.372.camel@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jintack Lim , xen-devel Cc: Ian Campbell List-Id: xen-devel@lists.xenproject.org On 03/09/15 17:06, Jintack Lim wrote: > Hi, Hi Jintack, > On Thu, Sep 3, 2015 at 11:34 AM, Ian Campbell > wrote: > > On Thu, 2015-09-03 at 10:59 -0400, Jintack Lim wrote: > > Hi, > > > > While I was running Apache server, > > I found that one of Dom0 vcpu is running 100% to handle irqs, > > and those irqs are set to be processed only on that specific vcpu. > > > > Referring to this document, > > http://wiki.xen.org/wiki/Network_Throughput_and_Performance_Guide > > I tried to change smp_affinity by writing a value to /proc/irq/ > -no>/smp_affinity, > > however the smp_affinity value was not changed. > > > > I'm working on Xen 4.5 on ARMv8, > > and the irq is netback tx interrupt. > > > > # cat /proc/irq/106/smp_affinity > > 1 > > > > # cat /proc/interrupts > > ... > > 106: 53849 0 0 0 xen-dyn-event > > vif1.1-q0-tx > > 107: 1 0 0 0 xen-dyn-event > > vif1.1-q0-rx > > 108: 61460 0 0 0 xen-dyn-event > > vif1.1-q1-tx > > 109: 1 0 0 0 xen-dyn-event > > vif1.1-q1-rx > > 110: 67118 0 0 0 xen-dyn-event > > vif1.1-q2-tx > > 111: 1 0 0 0 xen-dyn-event > > vif1.1-q2-rx > > 112: 58273 0 0 0 xen-dyn-event > > vif1.1-q3-tx > > 113: 1 0 0 0 xen-dyn-event > > vif1.1-q3-rx > > ... > > > > What would be the way to change smp_affinity? All those interrupts are in-fine event channels. This is a bug in Linux which I sent a fix a month ago and it's queued in xentip for Linux 4.3 [1] The event channel rebinding was not working because we don't have vector callback on ARM and therefore the driver was only allowing event channel to be routed on VCPU0. commit 4a5b69464e51f4a8dd432e8c2a1468630df1a53c Author: Julien Grall Date: Tue Jul 28 10:10:42 2015 +0100 xen/events: Support event channel rebind on ARM Currently, the event channel rebind code is gated with the presence of the vector callback. The virtual interrupt controller on ARM has the concept of per-CPU interrupt (PPI) which allow us to support per-VCPU event channel.Therefore there is no need of vector callback for ARM. Xen is already using a free PPI to notify the guest VCPU of an event. Furthermore, the xen code initialization in Linux (see arch/arm/xen/enlighten.c) is requesting correctly a per-CPU IRQ. Introduce new helper xen_support_evtchn_rebind to allow architecture decide whether rebind an event is support or not. It will always return true on ARM and keep the same behavior on x86. This is also allow us to drop the usage of xen_have_vector_callback entirely in the ARM code. Signed-off-by: Julien Grall Signed-off-by: David Vrabel Regards, [1] https://lkml.org/lkml/2015/7/28/205 -- Julien Grall From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jintack Lim Subject: Re: [Xen-users] Changing netback tx interrupts affinity on Dom0 Date: Thu, 3 Sep 2015 17:12:01 -0400 Message-ID: References: <1441294496.26292.372.camel@citrix.com> <55E88FB7.6020004@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0761059272786387095==" Return-path: In-Reply-To: <55E88FB7.6020004@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Julien Grall Cc: Ian Campbell , xen-devel List-Id: xen-devel@lists.xenproject.org --===============0761059272786387095== Content-Type: multipart/alternative; boundary=089e013a239cf7a9b7051ede3bf8 --089e013a239cf7a9b7051ede3bf8 Content-Type: text/plain; charset=UTF-8 Thanks Julien. On Thu, Sep 3, 2015 at 2:21 PM, Julien Grall wrote: > On 03/09/15 17:06, Jintack Lim wrote: > > Hi, > > Hi Jintack, > > > On Thu, Sep 3, 2015 at 11:34 AM, Ian Campbell > > wrote: > > > > On Thu, 2015-09-03 at 10:59 -0400, Jintack Lim wrote: > > > Hi, > > > > > > While I was running Apache server, > > > I found that one of Dom0 vcpu is running 100% to handle irqs, > > > and those irqs are set to be processed only on that specific vcpu. > > > > > > Referring to this document, > > > http://wiki.xen.org/wiki/Network_Throughput_and_Performance_Guide > > > I tried to change smp_affinity by writing a value to /proc/irq/ > > -no>/smp_affinity, > > > however the smp_affinity value was not changed. > > > > > > I'm working on Xen 4.5 on ARMv8, > > > and the irq is netback tx interrupt. > > > > > > # cat /proc/irq/106/smp_affinity > > > 1 > > > > > > # cat /proc/interrupts > > > ... > > > 106: 53849 0 0 0 xen-dyn-event > > > vif1.1-q0-tx > > > 107: 1 0 0 0 xen-dyn-event > > > vif1.1-q0-rx > > > 108: 61460 0 0 0 xen-dyn-event > > > vif1.1-q1-tx > > > 109: 1 0 0 0 xen-dyn-event > > > vif1.1-q1-rx > > > 110: 67118 0 0 0 xen-dyn-event > > > vif1.1-q2-tx > > > 111: 1 0 0 0 xen-dyn-event > > > vif1.1-q2-rx > > > 112: 58273 0 0 0 xen-dyn-event > > > vif1.1-q3-tx > > > 113: 1 0 0 0 xen-dyn-event > > > vif1.1-q3-rx > > > ... > > > > > > What would be the way to change smp_affinity? > > All those interrupts are in-fine event channels. This is a bug in Linux > which I sent a fix a month ago and it's queued in xentip for Linux 4.3 [1] > > The event channel rebinding was not working because we don't have vector > callback on ARM and therefore the driver was only allowing event channel > to be routed on VCPU0. > > After applying this patch, I was able to set smp_affinity. In addition, the default value of smp_affinity for netback tx interrupts on Dom0 becomes "f" (I have four vcpus on Dom0). What I meant by 'netback tx interrupts' is vif1.1-q-tx This is slightly different from x86, because netback tx interrupts are pinned to separate vcpus on x86. I'm not sure this question is appropriate for devel-list, but why the policy to set smp_affinity for netback tx interrupts are different on ARM and x86? > commit 4a5b69464e51f4a8dd432e8c2a1468630df1a53c > Author: Julien Grall > Date: Tue Jul 28 10:10:42 2015 +0100 > > xen/events: Support event channel rebind on ARM > Currently, the event channel rebind code is gated with the presence of > the vector callback. > > The virtual interrupt controller on ARM has the concept of per-CPU > interrupt (PPI) which allow us to support per-VCPU event > channel.Therefore there is no need of vector callback for ARM. > > Xen is already using a free PPI to notify the guest VCPU of an event. > Furthermore, the xen code initialization in Linux (see > arch/arm/xen/enlighten.c) is requesting correctly a per-CPU IRQ. > > Introduce new helper xen_support_evtchn_rebind to allow architecture > decide whether rebind an event is support or not. It will always return > true on ARM and keep the same behavior on x86. > > This is also allow us to drop the usage of xen_have_vector_callback > entirely in the ARM code. > > Signed-off-by: Julien Grall > Signed-off-by: David Vrabel > > Regards, > > [1] https://lkml.org/lkml/2015/7/28/205 > > -- > Julien Grall > > --089e013a239cf7a9b7051ede3bf8 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Thanks Julien.

On Thu, Sep 3, 2015 at 2:21 PM, Julien Grall <julien.grall@citrix.com> = wrote:

On 03/09/15 17:06, Jintack Lim wrote:
> Hi,

Hi Jintack,

> On Thu, Sep 3, 2015 at 11:34 AM, Ian Campbell <ian.campb= ell@citrix.com

> <mailto:ian.campbell@citri= x.com>> wrote:
>
>=C2=A0 =C2=A0 =C2=A0On Thu, 2015-09-03 at 10:59 -0400, Jintack Lim wrot= e:
>=C2=A0 =C2=A0 =C2=A0> Hi,
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> While I was running Apache server,
>=C2=A0 =C2=A0 =C2=A0> I found that one of Dom0 vcpu is running 100% = to handle irqs,
>=C2=A0 =C2=A0 =C2=A0> and those irqs are set to be processed only on= that specific vcpu.
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> Referring to this document,
>=C2=A0 =C2=A0 =C2=A0> http:/= /wiki.xen.org/wiki/Network_Throughput_and_Performance_Guide
>=C2=A0 =C2=A0 =C2=A0> I tried to change smp_affinity by writing a va= lue to /proc/irq/<irq
>=C2=A0 =C2=A0 =C2=A0> -no>/smp_affinity,
>=C2=A0 =C2=A0 =C2=A0> however the smp_affinity value was not changed= .
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> I'm working on Xen 4.5 on ARMv8,
>=C2=A0 =C2=A0 =C2=A0> and the irq is netback tx interrupt.
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> # cat /proc/irq/106/smp_affinity
>=C2=A0 =C2=A0 =C2=A0> 1
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> # cat /proc/interrupts
>=C2=A0 =C2=A0 =C2=A0> ...
>=C2=A0 =C2=A0 =C2=A0> 106:=C2=A0 =C2=A0 =C2=A0 53849=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0xen-dyn-event
>=C2=A0 =C2=A0 =C2=A0> vif1.1-q0-tx
>=C2=A0 =C2=A0 =C2=A0> 107:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0xen-dyn-event
>=C2=A0 =C2=A0 =C2=A0> vif1.1-q0-rx
>=C2=A0 =C2=A0 =C2=A0> 108:=C2=A0 =C2=A0 =C2=A0 61460=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0xen-dyn-event
>=C2=A0 =C2=A0 =C2=A0> vif1.1-q1-tx
>=C2=A0 =C2=A0 =C2=A0> 109:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0xen-dyn-event
>=C2=A0 =C2=A0 =C2=A0> vif1.1-q1-rx
>=C2=A0 =C2=A0 =C2=A0> 110:=C2=A0 =C2=A0 =C2=A0 67118=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0xen-dyn-event
>=C2=A0 =C2=A0 =C2=A0> vif1.1-q2-tx
>=C2=A0 =C2=A0 =C2=A0> 111:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0xen-dyn-event
>=C2=A0 =C2=A0 =C2=A0> vif1.1-q2-rx
>=C2=A0 =C2=A0 =C2=A0> 112:=C2=A0 =C2=A0 =C2=A0 58273=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0xen-dyn-event
>=C2=A0 =C2=A0 =C2=A0> vif1.1-q3-tx
>=C2=A0 =C2=A0 =C2=A0> 113:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0xen-dyn-event
>=C2=A0 =C2=A0 =C2=A0> vif1.1-q3-rx
>=C2=A0 =C2=A0 =C2=A0> ...
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> What would be the way to change smp_affinity?<= br>

All those interrupts are in-fine event channels. This is a bug = in Linux
which I sent a fix a month ago and it's queued in xentip for Linux 4.3 = [1]

The event channel rebinding was not working because we don't have vecto= r
callback on ARM and therefore the driver was only allowing event channel to be routed on VCPU0.

After applying this patch, I was able = to set smp_affinity.

In addition, the default valu= e of smp_affinity for netback tx interrupts on Dom0 becomes "f" (= I have four vcpus on Dom0).

What I meant by 'netback tx inter= rupts' is=C2=A0vif1.1-q<num>-tx

This is slightly differ= ent from x86, because netback tx interrupts are pinned to separate vcpus on= x86.

I'm not sure this question is appropriate for devel-lis= t,

but why the policy to set smp_affinity for netback tx interrup= ts are different on ARM and x86?

=C2=A0

commit 4a5b69464e51f4a8dd432e8c2a1468630df1a53c
Author: Julien Grall <julien.grall@citrix.com>
Date:=C2=A0 =C2=A0Tue Jul 28 10:10:42 2015 +0100

xen/events: Support event channel rebind on ARM
Currently, the event channel rebind code is gated with the presence of
the vector callback.

The virtual interrupt controller on ARM has the concept of per-CPU
interrupt (PPI) which allow us to support per-VCPU event
channel.Therefore there is no need of vector callback for ARM.

Xen is already using a free PPI to notify the guest VCPU of an event.
Furthermore, the xen code initialization in Linux (see
arch/arm/xen/enlighten.c) is requesting correctly a per-CPU IRQ.

Introduce new helper xen_support_evtchn_rebind to allow architecture
decide whether rebind an event is support or not. It will always return
true on ARM and keep the same behavior on x86.

This is also allow us to drop the usage of xen_have_vector_callback
entirely in the ARM code.

Signed-off-by: Julien Grall <julien.grall@citrix.com>=
Signed-off-by: David Vrabel <david.vrabel@citrix.com>=

Regards,

[1] https://lkml.org/lkml/2015/7/28/205

--
Julien Grall

--089e013a239cf7a9b7051ede3bf8-- --===============0761059272786387095== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============0761059272786387095==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Julien Grall Subject: Re: [Xen-users] Changing netback tx interrupts affinity on Dom0 Date: Fri, 4 Sep 2015 14:28:18 +0100 Message-ID: <55E99C72.3030503@citrix.com> References: <1441294496.26292.372.camel@citrix.com> <55E88FB7.6020004@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Jintack Lim Cc: David Vrabel , Ian Campbell , xen-devel List-Id: xen-devel@lists.xenproject.org Hi, On 03/09/15 22:12, Jintack Lim wrote: > On Thu, Sep 3, 2015 at 2:21 PM, Julien Grall > wrote: > > On 03/09/15 17:06, Jintack Lim wrote: > > Hi, > > Hi Jintack, > > > On Thu, Sep 3, 2015 at 11:34 AM, Ian Campbell > > >> > wrote: > > > > On Thu, 2015-09-03 at 10:59 -0400, Jintack Lim wrote: > > > Hi, > > > > > > While I was running Apache server, > > > I found that one of Dom0 vcpu is running 100% to handle irqs, > > > and those irqs are set to be processed only on that specific > vcpu. > > > > > > Referring to this document, > > > > http://wiki.xen.org/wiki/Network_Throughput_and_Performance_Guide > > > I tried to change smp_affinity by writing a value to > /proc/irq/ > > -no>/smp_affinity, > > > however the smp_affinity value was not changed. > > > > > > I'm working on Xen 4.5 on ARMv8, > > > and the irq is netback tx interrupt. > > > > > > # cat /proc/irq/106/smp_affinity > > > 1 > > > > > > # cat /proc/interrupts > > > ... > > > 106: 53849 0 0 0 xen-dyn-event > > > vif1.1-q0-tx > > > 107: 1 0 0 0 xen-dyn-event > > > vif1.1-q0-rx > > > 108: 61460 0 0 0 xen-dyn-event > > > vif1.1-q1-tx > > > 109: 1 0 0 0 xen-dyn-event > > > vif1.1-q1-rx > > > 110: 67118 0 0 0 xen-dyn-event > > > vif1.1-q2-tx > > > 111: 1 0 0 0 xen-dyn-event > > > vif1.1-q2-rx > > > 112: 58273 0 0 0 xen-dyn-event > > > vif1.1-q3-tx > > > 113: 1 0 0 0 xen-dyn-event > > > vif1.1-q3-rx > > > ... > > > > > > What would be the way to change smp_affinity? > > All those interrupts are in-fine event channels. This is a bug in Linux > which I sent a fix a month ago and it's queued in xentip for Linux > 4.3 [1] > > The event channel rebinding was not working because we don't have vector > callback on ARM and therefore the driver was only allowing event channel > to be routed on VCPU0. > > > After applying this patch, I was able to set smp_affinity. > > In addition, the default value of smp_affinity for netback tx interrupts > on Dom0 becomes "f" (I have four vcpus on Dom0). > What I meant by 'netback tx interrupts' is vif1.1-q-tx > This is slightly different from x86, because netback tx interrupts are > pinned to separate vcpus on x86. > I'm not sure this question is appropriate for devel-list, > but why the policy to set smp_affinity for netback tx interrupts are > different on ARM and x86? I'm not sure why. By default on both ARM and x86 the IRQ code is setting the affinity to all the vCPU. Note that you are likely using PV for x86 DOM0 which is different than HVM so it may be possible that we differ for setting the affinity. I've CCed David who was working on the event channel code. It may have more insights. Regards, -- Julien Grall