From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrey Grodzovsky Subject: Re: Issues with PCI-Passtrough (VT-d) in HVM with Xen 4.6 Date: Fri, 17 Jun 2016 23:24:33 -0400 Message-ID: References: <576170CA02000078000F5556@prv-mh.provo.novell.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3996762443705372354==" Return-path: In-Reply-To: <576170CA02000078000F5556@prv-mh.provo.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Jan Beulich Cc: =?UTF-8?B?SsO8cmdlbiBXYWx0ZXIg4oCiIFF1YXR0cnU=?= , xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org --===============3996762443705372354== Content-Type: multipart/alternative; boundary=94eb2c07ff8092537b0535850204 --94eb2c07ff8092537b0535850204 Content-Type: text/plain; charset=UTF-8 On Wed, Jun 15, 2016 at 9:14 AM, Jan Beulich wrote: > >>> On 15.06.16 at 12:45, wrote: > > In reply to - > > http://lists.xen.org/archives/html/xen-devel/2016-06/msg00622.html > > > > HI, I am working with Jurgen on the issue, as per Jan's request I tried > to > > write explicitly only latency timer to be written - > > bool force_write = false; > > if ((dev_data->permissive || xen_pcibk_permissive) && > > offset == PCI_CACHE_LINE_SIZE && size == 4) > > force_write = true; > > ... > > > > if ((force_write || !handled) && !err) {...} > > > > But then it exposed another issue, the command register field seems not > to > > be restored also > > because I think the bits which are to be restored are not > > in PCI_COMMAND_GUEST mask. > > Please be more precise: Which bits in particular are not getting > set back to the needed values (I would guess the memory > and/or I/O decode ones, which we specifically must not allow > the guest to control, but I'd like to be certain)? > Indeed, 0x103 which would be MEMORY,IO and SERR. > > If my guess is correct, then I think rather than adding some > hackish workaround to pciback you'd better see whether there's > a way to cause a pci_disable_device() through your driver before > the reset, and a pci_enable_device() after. Or actually, I think > you can do this via plain config space writes from your driver: Try > writing the Command Register with the two decode bits clear > right before restoring the intended value (see the logic close to > the top of command_write()). > > Jan > > So i tried the first option, as you suggested by writing to /sys/bus/pci/devices/0000\:00\:00.0/enable '0' == disable before reset and '1' == enable after from the test app, but no writes get through to pic-back on DOM0 , only the reads. But this basically still tries to write the MEMORY and IO bits to command register down the call stack in pci_enable_resources which would again be blocked in xen-pciback xen_pcibk_config_write or am I missing something ? --94eb2c07ff8092537b0535850204 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

= On Wed, Jun 15, 2016 at 9:14 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>> On 15.06.16 = at 12:45, <andrey2805@gmail.com<= /a>> wrote:
> In reply to -
> http://lists.xen.org/archives= /html/xen-devel/2016-06/msg00622.html
>
> HI, I am working with Jurgen on the issue, as per Jan's request I = tried to
> write explicitly only latency timer to be written -
> bool force_write =3D false;
> if ((dev_data->permissive || xen_pcibk_permissive) &&
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0offset =3D=3D PC= I_CACHE_LINE_SIZE && size =3D=3D 4)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 force_write =3D true;<= br> > ...
>
> if ((force_write || !handled) && !err) {...}
>
> But then it exposed another issue, the command register field seems no= t to
> be restored also
> because I think the bits which are to be restored are not
> in PCI_COMMAND_GUEST mask.

Please be more precise: Which bits in particular are not getting
set back to the needed values (I would guess the memory
and/or I/O decode ones, which we specifically must not allow
the guest to control, but I'd like to be certain)?
=C2=A0
Indeed, 0x103 which would be =C2=A0MEMORY,IO and SERR.

If my guess is correct, then I think rather than adding some
hackish workaround to pciback you'd better see whether there's
a way to cause a pci_disable_device() through your driver before
the reset, and a pci_enable_device() after. Or actually, I think
you can do this via plain config space writes from your driver: Try
writing the Command Register with the two decode bits clear
right before restoring the intended value (see the logic close to
the top of command_write()).

Jan

So i tried the first option, as you sug= gested by writing to=C2=A0/sys/bus/pci/devices/0000\:00\:00.0/enable
<= div>'0' =3D=3D disable before reset and '1' =3D=3D enable a= fter from the test app, but no writes get through to pic-back on DOM0 , onl= y the reads.
But this basically still tries to write the MEMORY a= nd IO bits to command register down the call stack in=C2=A0pci_enable_resources=C2=A0
which would again be b= locked in xen-pciback=C2=A0xen_pcibk_config_write or am I missing something= ?

--94eb2c07ff8092537b0535850204-- --===============3996762443705372354== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwOi8vbGlzdHMueGVuLm9y Zy94ZW4tZGV2ZWwK --===============3996762443705372354==--