IOMMU and AMD 890fx

IOMMU and AMD 890fx

[Leonardo Prosperi]

[-- Attachment #1.1: Type: text/plain, Size: 245 bytes --]

Hi,
All the reviews for the AMD 890fx chipset reported that it supports IOMMU
v1.2.
But I cannot find any mention of that on AMD site.

Does 890fx support IOMMU?
Do 890fx motherboards with IOMMU exist?
Does Xen support 890fx?

Regards,
Leonardo

[-- Attachment #1.2: Type: text/html, Size: 319 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: IOMMU and AMD 890fx

[Huang2, Wei]

[-- Attachment #1.1: Type: text/plain, Size: 883 bytes --]

Leonardo,

I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly mentioned by external sites, such as http://techreport.com/articles.x/18825. It looks like Asus, Gigabyte and MSI will ship motherboards with 890FX chipset. Xen 4.0 fully supports AMD IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS table). Please pay a bit attention to this area.

-Wei

From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo Prosperi
Sent: Sunday, May 09, 2010 1:02 PM
To: xen-devel@lists.xensource.com
Subject: [Xen-devel] IOMMU and AMD 890fx


Hi,
All the reviews for the AMD 890fx chipset reported that it supports IOMMU v1.2.
But I cannot find any mention of that on AMD site.

Does 890fx support IOMMU?
Do 890fx motherboards with IOMMU exist?
Does Xen support 890fx?

Regards,
Leonardo


[-- Attachment #1.2: Type: text/html, Size: 4033 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: IOMMU and AMD 890fx

[Pasi Kärkkäinen]

On Sun, May 09, 2010 at 11:43:04PM -0500, Huang2, Wei wrote:
>    Leonardo,
> 
> 
> 
>    I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly
>    mentioned by external sites, such as
>    [1]http://techreport.com/articles.x/18825. It looks like Asus, Gigabyte
>    and MSI will ship motherboards with 890FX chipset. Xen 4.0 fully supports
>    AMD IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS table).
>    Please pay a bit attention to this area.
> 

Thanks, I'll add this info to the wiki later today.

-- Pasi

> 
> 
>    -Wei
> 
> 
> 
>    From: xen-devel-bounces@lists.xensource.com
>    [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo
>    Prosperi
>    Sent: Sunday, May 09, 2010 1:02 PM
>    To: xen-devel@lists.xensource.com
>    Subject: [Xen-devel] IOMMU and AMD 890fx
> 
> 
> 
> 
> 
>    Hi,
>    All the reviews for the AMD 890fx chipset reported that it supports IOMMU
>    v1.2.
>    But I cannot find any mention of that on AMD site.
> 
>    Does 890fx support IOMMU?
>    Do 890fx motherboards with IOMMU exist?
>    Does Xen support 890fx?
> 
>    Regards,
>    Leonardo
> 
> 
> 
> References
> 
>    Visible links
>    1. http://techreport.com/articles.x/18825

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: IOMMU and AMD 890fx

[Sander Eikelenboom]

Hello Wei,

I hope motherboard manufacturers do a better job with supporting this iommu in the bios than they did with intel's. Especially Asus made a mess of it, and hasn't fixed it in newer bioses for multiple motherboards.
And since all motherboard reviews don't mention nor test the iommu it's hard to know upfront if it's supported :-(

--

Sander

> Leonardo,

> I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly mentioned by external sites, such as http://techreport.com/articles.x/18825. It looks like Asus, Gigabyte and MSI will ship motherboards with 890FX chipset. Xen 4.0 fully supports AMD IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS table). Please pay a bit attention to this area.

> -Wei

> From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo Prosperi
> Sent: Sunday, May 09, 2010 1:02 PM
> To: xen-devel@lists.xensource.com
> Subject: [Xen-devel] IOMMU and AMD 890fx


> Hi,
> All the reviews for the AMD 890fx chipset reported that it supports IOMMU v1.2.
> But I cannot find any mention of that on AMD site.

> Does 890fx support IOMMU?
> Do 890fx motherboards with IOMMU exist?
> Does Xen support 890fx?

> Regards,
> Leonardo




-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it

Re: IOMMU and AMD 890fx

[Leonardo Prosperi]

[-- Attachment #1.1: Type: text/plain, Size: 1789 bytes --]

Gigabyte motherboards don't support IOMMU:
http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/

MSI boards have the same problems:
http://comments.gmane.org/gmane.comp.emulators.xen.user/58630

ASUS Crosshair IV Formula has IOMMU, but I cannot find if it works with Xen.




On Sun, May 9, 2010 at 11:38 PM, Sander Eikelenboom <linux@eikelenboom.it>wrote:

> Hello Wei,
>
> I hope motherboard manufacturers do a better job with supporting this iommu
> in the bios than they did with intel's. Especially Asus made a mess of it,
> and hasn't fixed it in newer bioses for multiple motherboards.
> And since all motherboard reviews don't mention nor test the iommu it's
> hard to know upfront if it's supported :-(
>
> --
>
> Sander
>
> > Leonardo,
>
> > I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly
> mentioned by external sites, such as
> http://techreport.com/articles.x/18825. It looks like Asus, Gigabyte and
> MSI will ship motherboards with 890FX chipset. Xen 4.0 fully supports AMD
> IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS table).
> Please pay a bit attention to this area.
>
> > -Wei
>
> > From: xen-devel-bounces@lists.xensource.com [mailto:
> xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo Prosperi
> > Sent: Sunday, May 09, 2010 1:02 PM
> > To: xen-devel@lists.xensource.com
> > Subject: [Xen-devel] IOMMU and AMD 890fx
>
>
> > Hi,
> > All the reviews for the AMD 890fx chipset reported that it supports IOMMU
> v1.2.
> > But I cannot find any mention of that on AMD site.
>
> > Does 890fx support IOMMU?
> > Do 890fx motherboards with IOMMU exist?
> > Does Xen support 890fx?
>
> > Regards,
> > Leonardo
>
>
>
>
> --
> Best regards,
>  Sander                            mailto:linux@eikelenboom.it
>
>

[-- Attachment #1.2: Type: text/html, Size: 2792 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: IOMMU and AMD 890fx

[Sander Eikelenboom]

Thx for the forum threads, i have changed the wiki about vt-d/iommu to reflect these new experiences.

I was hoping AMD would be able to do a better job than Intel, and have learned from the BIOS missery Intel has with there VT-d, and instruct/help the manafacturers getting their bioses right.
It's a pity that a nice feature isn't supported. No sense in upgrading my setup then to a six core and mobo with IOMMU. :(


--
Sander



> Gigabyte motherboards don't support IOMMU:
> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/

> MSI boards have the same problems:
> http://comments.gmane.org/gmane.comp.emulators.xen.user/58630

> ASUS Crosshair IV Formula has IOMMU, but I cannot find if it works with Xen.




> On Sun, May 9, 2010 at 11:38 PM, Sander Eikelenboom <linux@eikelenboom.it>wrote:

>> Hello Wei,
>>
>> I hope motherboard manufacturers do a better job with supporting this iommu
>> in the bios than they did with intel's. Especially Asus made a mess of it,
>> and hasn't fixed it in newer bioses for multiple motherboards.
>> And since all motherboard reviews don't mention nor test the iommu it's
>> hard to know upfront if it's supported :-(
>>
>> --
>>
>> Sander
>>
>> > Leonardo,
>>
>> > I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly
>> mentioned by external sites, such as
>> http://techreport.com/articles.x/18825. It looks like Asus, Gigabyte and
>> MSI will ship motherboards with 890FX chipset. Xen 4.0 fully supports AMD
>> IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS table).
>> Please pay a bit attention to this area.
>>
>> > -Wei
>>
>> > From: xen-devel-bounces@lists.xensource.com [mailto:
>> xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo Prosperi
>> > Sent: Sunday, May 09, 2010 1:02 PM
>> > To: xen-devel@lists.xensource.com
>> > Subject: [Xen-devel] IOMMU and AMD 890fx
>>
>>
>> > Hi,
>> > All the reviews for the AMD 890fx chipset reported that it supports IOMMU
>> v1.2.
>> > But I cannot find any mention of that on AMD site.
>>
>> > Does 890fx support IOMMU?
>> > Do 890fx motherboards with IOMMU exist?
>> > Does Xen support 890fx?
>>
>> > Regards,
>> > Leonardo
>>
>>
>>
>>
>> --
>> Best regards,
>>  Sander                            mailto:linux@eikelenboom.it
>>
>>



-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it

Re: IOMMU and AMD 890fx

[Joop Boonen]

The ASRock 890FX Deluxe3 doesn't support the IOMMU technology too.
http://forums.amd.com/forum/messageview.cfm?catid=383&threadid=134410

I'm very disappointed that main board manufacturers don't enable all
features for the processor and main board.


On Thu, June 3, 2010 10:19 am, Sander Eikelenboom wrote:
> Thx for the forum threads, i have changed the wiki about vt-d/iommu to
> reflect these new experiences.
>
> I was hoping AMD would be able to do a better job than Intel, and have
> learned from the BIOS missery Intel has with there VT-d, and instruct/help
> the manafacturers getting their bioses right.

I have a feeling that it's not AMD but the main board manufacturers who
don't enable IOMMU in the BIOS. (It might be a hidden feature, for people
who want to hack the BIOS at your own risk).
http://www.rebelshavenforum.com/sis-bin/ultimatebb.cgi

> It's a pity that a nice feature isn't supported. No sense in upgrading my
> setup then to a six core and mobo with IOMMU. :(
>
>
> --
> Sander
>
>
>
>> Gigabyte motherboards don't support IOMMU:
>> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/
>
>> MSI boards have the same problems:
>> http://comments.gmane.org/gmane.comp.emulators.xen.user/58630
>
>> ASUS Crosshair IV Formula has IOMMU, but I cannot find if it works with
>> Xen.
>
>
>
>
>> On Sun, May 9, 2010 at 11:38 PM, Sander Eikelenboom
>> <linux@eikelenboom.it>wrote:
>
>>> Hello Wei,
>>>
>>> I hope motherboard manufacturers do a better job with supporting this
>>> iommu
>>> in the bios than they did with intel's. Especially Asus made a mess of
>>> it,
>>> and hasn't fixed it in newer bioses for multiple motherboards.
>>> And since all motherboard reviews don't mention nor test the iommu it's
>>> hard to know upfront if it's supported :-(
>>>
>>> --
>>>
>>> Sander
>>>
>>> > Leonardo,
>>>
>>> > I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly
>>> mentioned by external sites, such as
>>> http://techreport.com/articles.x/18825. It looks like Asus, Gigabyte
>>> and
>>> MSI will ship motherboards with 890FX chipset. Xen 4.0 fully supports
>>> AMD
>>> IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS table).
>>> Please pay a bit attention to this area.
>>>
>>> > -Wei
>>>
>>> > From: xen-devel-bounces@lists.xensource.com [mailto:
>>> xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo Prosperi
>>> > Sent: Sunday, May 09, 2010 1:02 PM
>>> > To: xen-devel@lists.xensource.com
>>> > Subject: [Xen-devel] IOMMU and AMD 890fx
>>>
>>>
>>> > Hi,
>>> > All the reviews for the AMD 890fx chipset reported that it supports
>>> IOMMU
>>> v1.2.
>>> > But I cannot find any mention of that on AMD site.
>>>
>>> > Does 890fx support IOMMU?
>>> > Do 890fx motherboards with IOMMU exist?
>>> > Does Xen support 890fx?
>>>
>>> > Regards,
>>> > Leonardo
>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>>  Sander                            mailto:linux@eikelenboom.it
>>>
>>>
>
>
>
> --
> Best regards,
>  Sander                            mailto:linux@eikelenboom.it
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Re: IOMMU and AMD 890fx

[Sander Eikelenboom]

Thursday, June 3, 2010, 10:53:29 AM, you wrote:

> The ASRock 890FX Deluxe3 doesn't support the IOMMU technology too.
> http://forums.amd.com/forum/messageview.cfm?catid=383&threadid=134410

> I'm very disappointed that main board manufacturers don't enable all
> features for the processor and main board.


> On Thu, June 3, 2010 10:19 am, Sander Eikelenboom wrote:
>> Thx for the forum threads, i have changed the wiki about vt-d/iommu to
>> reflect these new experiences.
>>
>> I was hoping AMD would be able to do a better job than Intel, and have
>> learned from the BIOS missery Intel has with there VT-d, and instruct/help
>> the manafacturers getting their bioses right.

> I have a feeling that it's not AMD but the main board manufacturers who
> don't enable IOMMU in the BIOS. (It might be a hidden feature, for people
> who want to hack the BIOS at your own risk).
> http://www.rebelshavenforum.com/sis-bin/ultimatebb.cgi

Ofcourse you are right, but as chipset manufacturer I would be pretty pissed and contact them.
Also I don't know how hard AMD and Intel make it to implement the feature in the bios. Since most of the time it's or disabled or broken.
On the other hand, they only support "microsoft windows" and that doesn't do a lot with an IOMMU. So in essence it's an unsupported feature.

It's too bad though, it seemed to be a very nice virtualization platform, 6 cores, iommu, affordable ...

Hmm guess i'm going to write every manufacturer if they supported, and/or if there are plans to support it.
Hopefully one will do it in the end ...

--
Sander

>> It's a pity that a nice feature isn't supported. No sense in upgrading my
>> setup then to a six core and mobo with IOMMU. :(
>>
>>
>> --
>> Sander
>>
>>
>>
>>> Gigabyte motherboards don't support IOMMU:
>>> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/
>>
>>> MSI boards have the same problems:
>>> http://comments.gmane.org/gmane.comp.emulators.xen.user/58630
>>
>>> ASUS Crosshair IV Formula has IOMMU, but I cannot find if it works with
>>> Xen.
>>
>>
>>
>>
>>> On Sun, May 9, 2010 at 11:38 PM, Sander Eikelenboom
>>> <linux@eikelenboom.it>wrote:
>>
>>>> Hello Wei,
>>>>
>>>> I hope motherboard manufacturers do a better job with supporting this
>>>> iommu
>>>> in the bios than they did with intel's. Especially Asus made a mess of
>>>> it,
>>>> and hasn't fixed it in newer bioses for multiple motherboards.
>>>> And since all motherboard reviews don't mention nor test the iommu it's
>>>> hard to know upfront if it's supported :-(
>>>>
>>>> --
>>>>
>>>> Sander
>>>>
>>>> > Leonardo,
>>>>
>>>> > I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly
>>>> mentioned by external sites, such as
>>>> http://techreport.com/articles.x/18825. It looks like Asus, Gigabyte
>>>> and
>>>> MSI will ship motherboards with 890FX chipset. Xen 4.0 fully supports
>>>> AMD
>>>> IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS table).
>>>> Please pay a bit attention to this area.
>>>>
>>>> > -Wei
>>>>
>>>> > From: xen-devel-bounces@lists.xensource.com [mailto:
>>>> xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo Prosperi
>>>> > Sent: Sunday, May 09, 2010 1:02 PM
>>>> > To: xen-devel@lists.xensource.com
>>>> > Subject: [Xen-devel] IOMMU and AMD 890fx
>>>>
>>>>
>>>> > Hi,
>>>> > All the reviews for the AMD 890fx chipset reported that it supports
>>>> IOMMU
>>>> v1.2.
>>>> > But I cannot find any mention of that on AMD site.
>>>>
>>>> > Does 890fx support IOMMU?
>>>> > Do 890fx motherboards with IOMMU exist?
>>>> > Does Xen support 890fx?
>>>>
>>>> > Regards,
>>>> > Leonardo
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>>  Sander                            mailto:linux@eikelenboom.it
>>>>
>>>>
>>
>>
>>
>> --
>> Best regards,
>>  Sander                            mailto:linux@eikelenboom.it
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>





-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

I got this answer from asrock support:

<answer>
So, I get more/final answer from Taiwan department:

For H/W part:
>From chipset spec, RD890(890FX) did support IOMMU.
But IOMMU also need CPU to support. Till now, AMD has no plan to support
IOMMU on desktop CPU (AM3 socket).
It only support with server CPU.

For BIOS part:
AMD has removed IOMMU related registers.
So there is no chance to let IOMMU work on our 890FX board unless AMD can
provide such a process, and registers.

Best regards,
ASRock Support
</answer>

So they say it is AMD's fault. :-(

Regards,
  Jens



On Thu, 3 Jun 2010 11:23:10 +0200, Sander Eikelenboom
<linux@eikelenboom.it> wrote:
> Thursday, June 3, 2010, 10:53:29 AM, you wrote:
> 
>> The ASRock 890FX Deluxe3 doesn't support the IOMMU technology too.
>> http://forums.amd.com/forum/messageview.cfm?catid=383&threadid=134410
> 
>> I'm very disappointed that main board manufacturers don't enable all
>> features for the processor and main board.
> 
> 
>> On Thu, June 3, 2010 10:19 am, Sander Eikelenboom wrote:
>>> Thx for the forum threads, i have changed the wiki about vt-d/iommu to
>>> reflect these new experiences.
>>>
>>> I was hoping AMD would be able to do a better job than Intel, and have
>>> learned from the BIOS missery Intel has with there VT-d, and
>>> instruct/help
>>> the manafacturers getting their bioses right.
> 
>> I have a feeling that it's not AMD but the main board manufacturers who
>> don't enable IOMMU in the BIOS. (It might be a hidden feature, for
people
>> who want to hack the BIOS at your own risk).
>> http://www.rebelshavenforum.com/sis-bin/ultimatebb.cgi
> 
> Ofcourse you are right, but as chipset manufacturer I would be pretty
> pissed and contact them.
> Also I don't know how hard AMD and Intel make it to implement the
feature
> in the bios. Since most of the time it's or disabled or broken.
> On the other hand, they only support "microsoft windows" and that
doesn't
> do a lot with an IOMMU. So in essence it's an unsupported feature.
> 
> It's too bad though, it seemed to be a very nice virtualization
platform,
> 6 cores, iommu, affordable ...
> 
> Hmm guess i'm going to write every manufacturer if they supported,
and/or
> if there are plans to support it.
> Hopefully one will do it in the end ...
> 
> --
> Sander
> 
>>> It's a pity that a nice feature isn't supported. No sense in upgrading
>>> my
>>> setup then to a six core and mobo with IOMMU. :(
>>>
>>>
>>> --
>>> Sander
>>>
>>>
>>>
>>>> Gigabyte motherboards don't support IOMMU:
>>>>
http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/
>>>
>>>> MSI boards have the same problems:
>>>> http://comments.gmane.org/gmane.comp.emulators.xen.user/58630
>>>
>>>> ASUS Crosshair IV Formula has IOMMU, but I cannot find if it works
with
>>>> Xen.
>>>
>>>
>>>
>>>
>>>> On Sun, May 9, 2010 at 11:38 PM, Sander Eikelenboom
>>>> <linux@eikelenboom.it>wrote:
>>>
>>>>> Hello Wei,
>>>>>
>>>>> I hope motherboard manufacturers do a better job with supporting
this
>>>>> iommu
>>>>> in the bios than they did with intel's. Especially Asus made a mess
of
>>>>> it,
>>>>> and hasn't fixed it in newer bioses for multiple motherboards.
>>>>> And since all motherboard reviews don't mention nor test the iommu
>>>>> it's
>>>>> hard to know upfront if it's supported :-(
>>>>>
>>>>> --
>>>>>
>>>>> Sander
>>>>>
>>>>> > Leonardo,
>>>>>
>>>>> > I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly
>>>>> mentioned by external sites, such as
>>>>> http://techreport.com/articles.x/18825. It looks like Asus, Gigabyte
>>>>> and
>>>>> MSI will ship motherboards with 890FX chipset. Xen 4.0 fully
supports
>>>>> AMD
>>>>> IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS
table).
>>>>> Please pay a bit attention to this area.
>>>>>
>>>>> > -Wei
>>>>>
>>>>> > From: xen-devel-bounces@lists.xensource.com [mailto:
>>>>> xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo
Prosperi
>>>>> > Sent: Sunday, May 09, 2010 1:02 PM
>>>>> > To: xen-devel@lists.xensource.com
>>>>> > Subject: [Xen-devel] IOMMU and AMD 890fx
>>>>>
>>>>>
>>>>> > Hi,
>>>>> > All the reviews for the AMD 890fx chipset reported that it
supports
>>>>> IOMMU
>>>>> v1.2.
>>>>> > But I cannot find any mention of that on AMD site.
>>>>>
>>>>> > Does 890fx support IOMMU?
>>>>> > Do 890fx motherboards with IOMMU exist?
>>>>> > Does Xen support 890fx?
>>>>>
>>>>> > Regards,
>>>>> > Leonardo
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>>  Sander                            mailto:linux@eikelenboom.it
>>>>>
>>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>>  Sander                            mailto:linux@eikelenboom.it
>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>>>

Re: IOMMU and AMD 890fx

[Sander Eikelenboom]

So in principle they would like to support it , but they can't / don't know how.
Very interested at AMD's answer then, unfortunately Wei Huang seems to be on vacation until June 7th.

For what i understood in the past was that AMD's solution would not depend on cpu support (as intel's vt-d does).
And with vga passthrough, iommu on the desktop looks like a good use of an iommu in my opinion.
--
Sander


Thursday, June 3, 2010, 12:51:23 PM, you wrote:


> I got this answer from asrock support:



> <answer>

> So, I get more/final answer from Taiwan department:



> For H/W part:

> From chipset spec, RD890(890FX) did support IOMMU.

> But IOMMU also need CPU to support. Till now, AMD has no plan to support

> IOMMU on desktop CPU (AM3 socket).

> It only support with server CPU.



> For BIOS part:

> AMD has removed IOMMU related registers.

> So there is no chance to let IOMMU work on our 890FX board unless AMD can

> provide such a process, and registers.



> Best regards,

> ASRock Support

> </answer>



> So they say it is AMD's fault. :-(



> Regards,

>   Jens







> On Thu, 3 Jun 2010 11:23:10 +0200, Sander Eikelenboom

> <linux@eikelenboom.it> wrote:

>> Thursday, June 3, 2010, 10:53:29 AM, you wrote:

>> 

>>> The ASRock 890FX Deluxe3 doesn't support the IOMMU technology too.

>>> http://forums.amd.com/forum/messageview.cfm?catid=383&threadid=134410

>> 

>>> I'm very disappointed that main board manufacturers don't enable all

>>> features for the processor and main board.

>> 

>> 

>>> On Thu, June 3, 2010 10:19 am, Sander Eikelenboom wrote:

>>>> Thx for the forum threads, i have changed the wiki about vt-d/iommu to

>>>> reflect these new experiences.

>>>>

>>>> I was hoping AMD would be able to do a better job than Intel, and have

>>>> learned from the BIOS missery Intel has with there VT-d, and

>>>> instruct/help

>>>> the manafacturers getting their bioses right.

>> 

>>> I have a feeling that it's not AMD but the main board manufacturers who

>>> don't enable IOMMU in the BIOS. (It might be a hidden feature, for

> people

>>> who want to hack the BIOS at your own risk).

>>> http://www.rebelshavenforum.com/sis-bin/ultimatebb.cgi

>> 

>> Ofcourse you are right, but as chipset manufacturer I would be pretty

>> pissed and contact them.

>> Also I don't know how hard AMD and Intel make it to implement the

> feature

>> in the bios. Since most of the time it's or disabled or broken.

>> On the other hand, they only support "microsoft windows" and that

> doesn't

>> do a lot with an IOMMU. So in essence it's an unsupported feature.

>> 

>> It's too bad though, it seemed to be a very nice virtualization

> platform,

>> 6 cores, iommu, affordable ...

>> 

>> Hmm guess i'm going to write every manufacturer if they supported,

> and/or

>> if there are plans to support it.

>> Hopefully one will do it in the end ...

>> 

>> --

>> Sander

>> 

>>>> It's a pity that a nice feature isn't supported. No sense in upgrading

>>>> my

>>>> setup then to a six core and mobo with IOMMU. :(

>>>>

>>>>

>>>> --

>>>> Sander

>>>>

>>>>

>>>>

>>>>> Gigabyte motherboards don't support IOMMU:

>>>>>

> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/

>>>>

>>>>> MSI boards have the same problems:

>>>>> http://comments.gmane.org/gmane.comp.emulators.xen.user/58630

>>>>

>>>>> ASUS Crosshair IV Formula has IOMMU, but I cannot find if it works

> with

>>>>> Xen.

>>>>

>>>>

>>>>

>>>>

>>>>> On Sun, May 9, 2010 at 11:38 PM, Sander Eikelenboom

>>>>> <linux@eikelenboom.it>wrote:

>>>>

>>>>>> Hello Wei,

>>>>>>

>>>>>> I hope motherboard manufacturers do a better job with supporting

> this

>>>>>> iommu

>>>>>> in the bios than they did with intel's. Especially Asus made a mess

> of

>>>>>> it,

>>>>>> and hasn't fixed it in newer bioses for multiple motherboards.

>>>>>> And since all motherboard reviews don't mention nor test the iommu

>>>>>> it's

>>>>>> hard to know upfront if it's supported :-(

>>>>>>

>>>>>> --

>>>>>>

>>>>>> Sander

>>>>>>

>>>>>> > Leonardo,

>>>>>>

>>>>>> > I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly

>>>>>> mentioned by external sites, such as

>>>>>> http://techreport.com/articles.x/18825. It looks like Asus, Gigabyte

>>>>>> and

>>>>>> MSI will ship motherboards with 890FX chipset. Xen 4.0 fully

> supports

>>>>>> AMD

>>>>>> IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS

> table).

>>>>>> Please pay a bit attention to this area.

>>>>>>

>>>>>> > -Wei

>>>>>>

>>>>>> > From: xen-devel-bounces@lists.xensource.com [mailto:

>>>>>> xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo

> Prosperi

>>>>>> > Sent: Sunday, May 09, 2010 1:02 PM

>>>>>> > To: xen-devel@lists.xensource.com

>>>>>> > Subject: [Xen-devel] IOMMU and AMD 890fx

>>>>>>

>>>>>>

>>>>>> > Hi,

>>>>>> > All the reviews for the AMD 890fx chipset reported that it

> supports

>>>>>> IOMMU

>>>>>> v1.2.

>>>>>> > But I cannot find any mention of that on AMD site.

>>>>>>

>>>>>> > Does 890fx support IOMMU?

>>>>>> > Do 890fx motherboards with IOMMU exist?

>>>>>> > Does Xen support 890fx?

>>>>>>

>>>>>> > Regards,

>>>>>> > Leonardo

>>>>>>

>>>>>>

>>>>>>

>>>>>>

>>>>>> --

>>>>>> Best regards,

>>>>>>  Sander                            mailto:linux@eikelenboom.it

>>>>>>

>>>>>>

>>>>

>>>>

>>>>

>>>> --

>>>> Best regards,

>>>>  Sander                            mailto:linux@eikelenboom.it

>>>>

>>>>

>>>> _______________________________________________

>>>> Xen-devel mailing list

>>>> Xen-devel@lists.xensource.com

>>>> http://lists.xensource.com/xen-devel

>>>>



-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it

Re: IOMMU and AMD 890fx

[Joerg Roedel]

Hello Jens,

On Thu, Jun 03, 2010 at 12:51:23PM +0200, Jens Krehbiel-Gräther wrote:
> 
> I got this answer from asrock support:
> 
> <answer>
> So, I get more/final answer from Taiwan department:
> 
> For H/W part:
> >From chipset spec, RD890(890FX) did support IOMMU.
> But IOMMU also need CPU to support.

Thats wrong. The IOMMU is a feature of the chipset northbridge and does
not need additional cpu support.

> Till now, AMD has no plan to support
> IOMMU on desktop CPU (AM3 socket).

There are AM3 boards out there with server a chipset that have it
enabled. Its no question of a socket.

> It only support with server CPU.

Also wrong. I have seen systems with desktop cpus that have IOMMU
enabled (Phenom X4 and Phenom X6).

> For BIOS part:

> AMD has removed IOMMU related registers.

Hard to believe. I have found people in the net who say that the ASUS
Crosshair IV mobo has IOMMU support in the BIOS. So if they can enable
it, asrock can do that too. Its just the BIOS which has to enable it

> So they say it is AMD's fault. :-(

I doubt that.

	Joerg

Re: IOMMU and AMD 890fx

[Sander Eikelenboom]

Hi Joerg,

When i look online in the manual for the  ASUS Crosshair IV, i don't see any mention of IOMMU.
Not in the speficication list, and not in the bios option screenshots (only the SVM option is mentioned).

Is there someone within AMD that has contact with motherboard manufacturers(and the bios makers) that could address this issue with them ?

--
Sander

Thursday, June 3, 2010, 1:26:49 PM, you wrote:

> Hello Jens,

> On Thu, Jun 03, 2010 at 12:51:23PM +0200, Jens Krehbiel-Gräther wrote:
>> 
>> I got this answer from asrock support:
>> 
>> <answer>
>> So, I get more/final answer from Taiwan department:
>> 
>> For H/W part:
>> >From chipset spec, RD890(890FX) did support IOMMU.
>> But IOMMU also need CPU to support.

> Thats wrong. The IOMMU is a feature of the chipset northbridge and does
> not need additional cpu support.

>> Till now, AMD has no plan to support
>> IOMMU on desktop CPU (AM3 socket).

> There are AM3 boards out there with server a chipset that have it
> enabled. Its no question of a socket.

>> It only support with server CPU.

> Also wrong. I have seen systems with desktop cpus that have IOMMU
> enabled (Phenom X4 and Phenom X6).

>> For BIOS part:

>> AMD has removed IOMMU related registers.

> Hard to believe. I have found people in the net who say that the ASUS
> Crosshair IV mobo has IOMMU support in the BIOS. So if they can enable
> it, asrock can do that too. Its just the BIOS which has to enable it

>> So they say it is AMD's fault. :-(

> I doubt that.

>         Joerg




-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it

Re: IOMMU and AMD 890fx

[Andre Przywara]

Hi,

Jörg is correct, thanks for picking this up. Some additions inline.
Joerg Roedel wrote:
> Hello Jens,
> 
> On Thu, Jun 03, 2010 at 12:51:23PM +0200, Jens Krehbiel-Gräther wrote:
>> I got this answer from asrock support:
>>
>> <answer>
>> So, I get more/final answer from Taiwan department:
>>
>> For H/W part:
>> >From chipset spec, RD890(890FX) did support IOMMU.
>> But IOMMU also need CPU to support.
> 
> Thats wrong. The IOMMU is a feature of the chipset northbridge and does
> not need additional cpu support.
> 
>> Till now, AMD has no plan to support
>> IOMMU on desktop CPU (AM3 socket).
> 
> There are AM3 boards out there with server a chipset that have it
> enabled. Its no question of a socket.
That would be the Tyan S8005, although the official CPU support list 
only lists Opteron parts (those in AM3 socket: Suzuka and Budapest)
> 
>> It only support with server CPU.
> 
> Also wrong. I have seen systems with desktop cpus that have IOMMU
> enabled (Phenom X4 and Phenom X6).
> 
>> For BIOS part:
> 
>> AMD has removed IOMMU related registers.
> 
> Hard to believe. I have found people in the net who say that the ASUS
> Crosshair IV mobo has IOMMU support in the BIOS. So if they can enable
> it, asrock can do that too. Its just the BIOS which has to enable it

If you want to check this yourself:
For out-of-the-box support there needs to be an ACPI table called IVRS 
setup by the BIOS, you can spot this in xm dmesg (or in dmesg on native 
Linux):
(XEN) ACPI: IVRS C7E7ACC8, 00E0 (r1  AMD     RD890S  6040000 AMD         0)
On a native Linux you can also find the table in /sys/firmware/acpi/tables
If the table is present, recent Xen versions (as well as Linux and KVM) 
should automatically enable and use the IOMMU.

If the ACPI table is missing, you can try to look for the corresponding 
PCI device: it should have a vendor/device-id of 1002:5a23, lspci lists 
it as:
00:00.2 Generic system peripheral [0806]: ATI Technologies Inc Device 5a23
If anyone has a board with the PCI device but a missing ACPI table, 
please tell us.

We are already trying to improve the situation, you could help us by 
reporting the IOMMU support status on 890FX boards. Simply check for the 
ACPI table:
# xm dmesg | grep IVRS    or
$ ls /sys/firmware/acpi/tables/IVRS
and by checking for the PCI device:
$ /sbin/lspci -d 1002:5a23


Hope that helps.

Regards,
Andre.


-- 
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448 3567 12
----to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Hi!

I know that iommu is only a chipset feature, but it seems that asrock
don't know that... :-(
I will have a look on it when I'm back home (sunday) and then check for
the ACPI tables and the pci device.
I only remember that therewas a kernel message when booting like

"Please activate IOMMU in the bios"

in fact when it is there on 890FX the asrock 890FX Deluxe3 hasn't enabled
it and there is no option for activation in the bios.
I will report on sunday.

Jens

On Thu, 3 Jun 2010 13:51:05 +0200, Andre Przywara <andre.przywara@amd.com>
wrote:
> Hi,
> 
> Jörg is correct, thanks for picking this up. Some additions inline.
> Joerg Roedel wrote:
>> Hello Jens,
>> 
>> On Thu, Jun 03, 2010 at 12:51:23PM +0200, Jens Krehbiel-Gräther wrote:
>>> I got this answer from asrock support:
>>>
>>> <answer>
>>> So, I get more/final answer from Taiwan department:
>>>
>>> For H/W part:
>>> >From chipset spec, RD890(890FX) did support IOMMU.
>>> But IOMMU also need CPU to support.
>> 
>> Thats wrong. The IOMMU is a feature of the chipset northbridge and does
>> not need additional cpu support.
>> 
>>> Till now, AMD has no plan to support
>>> IOMMU on desktop CPU (AM3 socket).
>> 
>> There are AM3 boards out there with server a chipset that have it
>> enabled. Its no question of a socket.
> That would be the Tyan S8005, although the official CPU support list 
> only lists Opteron parts (those in AM3 socket: Suzuka and Budapest)
>> 
>>> It only support with server CPU.
>> 
>> Also wrong. I have seen systems with desktop cpus that have IOMMU
>> enabled (Phenom X4 and Phenom X6).
>> 
>>> For BIOS part:
>> 
>>> AMD has removed IOMMU related registers.
>> 
>> Hard to believe. I have found people in the net who say that the ASUS
>> Crosshair IV mobo has IOMMU support in the BIOS. So if they can enable
>> it, asrock can do that too. Its just the BIOS which has to enable it
> 
> If you want to check this yourself:
> For out-of-the-box support there needs to be an ACPI table called IVRS 
> setup by the BIOS, you can spot this in xm dmesg (or in dmesg on native 
> Linux):
> (XEN) ACPI: IVRS C7E7ACC8, 00E0 (r1  AMD     RD890S  6040000 AMD        
0)
> On a native Linux you can also find the table in
/sys/firmware/acpi/tables
> If the table is present, recent Xen versions (as well as Linux and KVM) 
> should automatically enable and use the IOMMU.
> 
> If the ACPI table is missing, you can try to look for the corresponding 
> PCI device: it should have a vendor/device-id of 1002:5a23, lspci lists 
> it as:
> 00:00.2 Generic system peripheral [0806]: ATI Technologies Inc Device
5a23
> If anyone has a board with the PCI device but a missing ACPI table, 
> please tell us.
> 
> We are already trying to improve the situation, you could help us by 
> reporting the IOMMU support status on 890FX boards. Simply check for the

> ACPI table:
> # xm dmesg | grep IVRS    or
> $ ls /sys/firmware/acpi/tables/IVRS
> and by checking for the PCI device:
> $ /sbin/lspci -d 1002:5a23
> 
> 
> Hope that helps.
> 
> Regards,
> Andre.

Re: IOMMU and AMD 890fx

[Joerg Roedel]

On Thu, Jun 03, 2010 at 03:41:18PM +0200, Jens Krehbiel-Gräther wrote:
> "Please activate IOMMU in the bios"

This option refers to the GART, a very simple IOMMU in the processors
northbridge. It has nothing to do with the 890FX build-in IOMMU. So
don't get confused :)

	Joerg

RE: IOMMU and AMD 890fx

[Huang2, Wei]

[-- Attachment #1: Type: text/plain, Size: 6998 bytes --]

Normally AMD will release a common BIOS module/interface to OEM vendors, who then cherry-pick features to fit their products. I will be surprised if AMD specifically disables IOMMU feature for ASRock. 

Andre Przywara's email tells how to find out the IOMMU support in system (device 0:0.2 and ACPI IVRS table). Please follow it and tell us your findings.

From our side, we will contact BIOS team to find out the truth. In the meanwhile, it is necessary to compile a list of 890FX motherboards, which support IOMMU, in Xen wiki. So people won't find a surprise in future.

Regards,
-Wei 

-----Original Message-----
From: Sander Eikelenboom [mailto:linux@eikelenboom.it] 
Sent: Thursday, June 03, 2010 6:12 AM
To: Jens Krehbiel-Gräther
Cc: Joop Boonen; Leonardo Prosperi; Huang2, Wei; xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] IOMMU and AMD 890fx

So in principle they would like to support it , but they can't / don't know how.
Very interested at AMD's answer then, unfortunately Wei Huang seems to be on vacation until June 7th.

For what i understood in the past was that AMD's solution would not depend on cpu support (as intel's vt-d does).
And with vga passthrough, iommu on the desktop looks like a good use of an iommu in my opinion.
--
Sander


Thursday, June 3, 2010, 12:51:23 PM, you wrote:


> I got this answer from asrock support:



> <answer>

> So, I get more/final answer from Taiwan department:



> For H/W part:

> From chipset spec, RD890(890FX) did support IOMMU.

> But IOMMU also need CPU to support. Till now, AMD has no plan to support

> IOMMU on desktop CPU (AM3 socket).

> It only support with server CPU.



> For BIOS part:

> AMD has removed IOMMU related registers.

> So there is no chance to let IOMMU work on our 890FX board unless AMD can

> provide such a process, and registers.



> Best regards,

> ASRock Support

> </answer>



> So they say it is AMD's fault. :-(



> Regards,

>   Jens







> On Thu, 3 Jun 2010 11:23:10 +0200, Sander Eikelenboom

> <linux@eikelenboom.it> wrote:

>> Thursday, June 3, 2010, 10:53:29 AM, you wrote:

>> 

>>> The ASRock 890FX Deluxe3 doesn't support the IOMMU technology too.

>>> http://forums.amd.com/forum/messageview.cfm?catid=383&threadid=134410

>> 

>>> I'm very disappointed that main board manufacturers don't enable all

>>> features for the processor and main board.

>> 

>> 

>>> On Thu, June 3, 2010 10:19 am, Sander Eikelenboom wrote:

>>>> Thx for the forum threads, i have changed the wiki about vt-d/iommu to

>>>> reflect these new experiences.

>>>>

>>>> I was hoping AMD would be able to do a better job than Intel, and have

>>>> learned from the BIOS missery Intel has with there VT-d, and

>>>> instruct/help

>>>> the manafacturers getting their bioses right.

>> 

>>> I have a feeling that it's not AMD but the main board manufacturers who

>>> don't enable IOMMU in the BIOS. (It might be a hidden feature, for

> people

>>> who want to hack the BIOS at your own risk).

>>> http://www.rebelshavenforum.com/sis-bin/ultimatebb.cgi

>> 

>> Ofcourse you are right, but as chipset manufacturer I would be pretty

>> pissed and contact them.

>> Also I don't know how hard AMD and Intel make it to implement the

> feature

>> in the bios. Since most of the time it's or disabled or broken.

>> On the other hand, they only support "microsoft windows" and that

> doesn't

>> do a lot with an IOMMU. So in essence it's an unsupported feature.

>> 

>> It's too bad though, it seemed to be a very nice virtualization

> platform,

>> 6 cores, iommu, affordable ...

>> 

>> Hmm guess i'm going to write every manufacturer if they supported,

> and/or

>> if there are plans to support it.

>> Hopefully one will do it in the end ...

>> 

>> --

>> Sander

>> 

>>>> It's a pity that a nice feature isn't supported. No sense in upgrading

>>>> my

>>>> setup then to a six core and mobo with IOMMU. :(

>>>>

>>>>

>>>> --

>>>> Sander

>>>>

>>>>

>>>>

>>>>> Gigabyte motherboards don't support IOMMU:

>>>>>

> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/

>>>>

>>>>> MSI boards have the same problems:

>>>>> http://comments.gmane.org/gmane.comp.emulators.xen.user/58630

>>>>

>>>>> ASUS Crosshair IV Formula has IOMMU, but I cannot find if it works

> with

>>>>> Xen.

>>>>

>>>>

>>>>

>>>>

>>>>> On Sun, May 9, 2010 at 11:38 PM, Sander Eikelenboom

>>>>> <linux@eikelenboom.it>wrote:

>>>>

>>>>>> Hello Wei,

>>>>>>

>>>>>> I hope motherboard manufacturers do a better job with supporting

> this

>>>>>> iommu

>>>>>> in the bios than they did with intel's. Especially Asus made a mess

> of

>>>>>> it,

>>>>>> and hasn't fixed it in newer bioses for multiple motherboards.

>>>>>> And since all motherboard reviews don't mention nor test the iommu

>>>>>> it's

>>>>>> hard to know upfront if it's supported :-(

>>>>>>

>>>>>> --

>>>>>>

>>>>>> Sander

>>>>>>

>>>>>> > Leonardo,

>>>>>>

>>>>>> > I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly

>>>>>> mentioned by external sites, such as

>>>>>> http://techreport.com/articles.x/18825. It looks like Asus, Gigabyte

>>>>>> and

>>>>>> MSI will ship motherboards with 890FX chipset. Xen 4.0 fully

> supports

>>>>>> AMD

>>>>>> IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS

> table).

>>>>>> Please pay a bit attention to this area.

>>>>>>

>>>>>> > -Wei

>>>>>>

>>>>>> > From: xen-devel-bounces@lists.xensource.com [mailto:

>>>>>> xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo

> Prosperi

>>>>>> > Sent: Sunday, May 09, 2010 1:02 PM

>>>>>> > To: xen-devel@lists.xensource.com

>>>>>> > Subject: [Xen-devel] IOMMU and AMD 890fx

>>>>>>

>>>>>>

>>>>>> > Hi,

>>>>>> > All the reviews for the AMD 890fx chipset reported that it

> supports

>>>>>> IOMMU

>>>>>> v1.2.

>>>>>> > But I cannot find any mention of that on AMD site.

>>>>>>

>>>>>> > Does 890fx support IOMMU?

>>>>>> > Do 890fx motherboards with IOMMU exist?

>>>>>> > Does Xen support 890fx?

>>>>>>

>>>>>> > Regards,

>>>>>> > Leonardo

>>>>>>

>>>>>>

>>>>>>

>>>>>>

>>>>>> --

>>>>>> Best regards,

>>>>>>  Sander                            mailto:linux@eikelenboom.it

>>>>>>

>>>>>>

>>>>

>>>>

>>>>

>>>> --

>>>> Best regards,

>>>>  Sander                            mailto:linux@eikelenboom.it

>>>>

>>>>

>>>> _______________________________________________

>>>> Xen-devel mailing list

>>>> Xen-devel@lists.xensource.com

>>>> http://lists.xensource.com/xen-devel

>>>>



-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it



[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Hi!

Am 04.06.2010 08:14, schrieb Huang2, Wei:
> Normally AMD will release a common BIOS module/interface to OEM vendors, who then cherry-pick features to fit their products. I will be surprised if AMD specifically disables IOMMU feature for ASRock.
>
> Andre Przywara's email tells how to find out the IOMMU support in system (device 0:0.2 and ACPI IVRS table). Please follow it and tell us your findings.

Here is the output:

ls /sys/firmware/acpi/tables/IVRS
not possible because there is no IVRS in this directory. The only files 
there are: AAFT  APIC  DSDT  dynamic/  FACP  FACS  HPET  MCFG  OEMB 
SRAT  SSDT

lspci -d 1002:5a23
There is no device with this id.

The output of the kernel at boot time is:
root@xen-01:~# dmesg |grep -i iommu
[    0.000000] Please enable the IOMMU option in the BIOS setup
[    0.817689] PCI-DMA: using GART IOMMU.
[    0.817691] PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture
root@xen-01:~#


So this means that there is really no IOMMU support on this mainboard?

>  From our side, we will contact BIOS team to find out the truth. In the meanwhile, it is necessary to compile a list of 890FX motherboards, which support IOMMU, in Xen wiki. So people won't find a surprise in future.

I will be glad to see this list. Hope that there will be a few 
mainboards where it is possible to use IOMMU.

Jens

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Hi!

I ordered a Asus Crosshair Formula IV for testing. Will report next week 
if it works.

And I contacted Asrock support again and told them that Asus supports 
it. They will give the informations to their bios-team in Taiwan.

Meanwhile there is a very good and active discussion in the forum 
http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/index7.html 
about boards with 890fx and iommu.

Jens


Am 09.06.2010 22:26, schrieb Wei Huang:
> Hi,
>
> We talked with AMD client managers. They believe all 890FX platforms
> should be capable of supporting IOMMU. There must be mis-communication
> with motherboard vendors. We are in the process of contacting with them.
> Hopefully the vendors will turn on this feature soon. For now, please
> just purchase the motherboards with IOMMU enabled (I think Joerg pointed
> out ASUS at least. I can't verify it myself though.).
>
> Best,
> -Wei
>
> Sander Eikelenboom wrote:
>> So in principle they would like to support it , but they can't / don't
>> know how.
>> Very interested at AMD's answer then, unfortunately Wei Huang seems to
>> be on vacation until June 7th.
>>
>> For what i understood in the past was that AMD's solution would not
>> depend on cpu support (as intel's vt-d does).
>> And with vga passthrough, iommu on the desktop looks like a good use
>> of an iommu in my opinion.
>> --
>> Sander
>>
>>
>> Thursday, June 3, 2010, 12:51:23 PM, you wrote:
>>
>>
>>> I got this answer from asrock support:
>>
>>
>>
>>> <answer>
>>
>>> So, I get more/final answer from Taiwan department:
>>
>>
>>
>>> For H/W part:
>>
>>> From chipset spec, RD890(890FX) did support IOMMU.
>>
>>> But IOMMU also need CPU to support. Till now, AMD has no plan to support
>>
>>> IOMMU on desktop CPU (AM3 socket).
>>
>>> It only support with server CPU.
>>
>>
>>
>>> For BIOS part:
>>
>>> AMD has removed IOMMU related registers.
>>
>>> So there is no chance to let IOMMU work on our 890FX board unless AMD
>>> can
>>
>>> provide such a process, and registers.
>>
>>
>>
>>> Best regards,
>>
>>> ASRock Support
>>
>>> </answer>
>>
>>
>>
>>> So they say it is AMD's fault. :-(
>>
>>
>>
>>> Regards,
>>
>>> Jens
>>
>>
>>
>>
>>
>>
>>
>>> On Thu, 3 Jun 2010 11:23:10 +0200, Sander Eikelenboom
>>
>>> <linux@eikelenboom.it> wrote:
>>
>>>> Thursday, June 3, 2010, 10:53:29 AM, you wrote:
>>
>>
>>>>> The ASRock 890FX Deluxe3 doesn't support the IOMMU technology too.
>>
>>>>> http://forums.amd.com/forum/messageview.cfm?catid=383&threadid=134410
>>
>>
>>>>> I'm very disappointed that main board manufacturers don't enable all
>>
>>>>> features for the processor and main board.
>>
>>
>>
>>>>> On Thu, June 3, 2010 10:19 am, Sander Eikelenboom wrote:
>>
>>>>>> Thx for the forum threads, i have changed the wiki about
>>>>>> vt-d/iommu to
>>
>>>>>> reflect these new experiences.
>>
>>
>>>>>> I was hoping AMD would be able to do a better job than Intel, and
>>>>>> have
>>
>>>>>> learned from the BIOS missery Intel has with there VT-d, and
>>
>>>>>> instruct/help
>>
>>>>>> the manafacturers getting their bioses right.
>>
>>
>>>>> I have a feeling that it's not AMD but the main board manufacturers
>>>>> who
>>
>>>>> don't enable IOMMU in the BIOS. (It might be a hidden feature, for
>>
>>> people
>>
>>>>> who want to hack the BIOS at your own risk).
>>
>>>>> http://www.rebelshavenforum.com/sis-bin/ultimatebb.cgi
>>
>>
>>>> Ofcourse you are right, but as chipset manufacturer I would be pretty
>>
>>>> pissed and contact them.
>>
>>>> Also I don't know how hard AMD and Intel make it to implement the
>>
>>> feature
>>
>>>> in the bios. Since most of the time it's or disabled or broken.
>>
>>>> On the other hand, they only support "microsoft windows" and that
>>
>>> doesn't
>>
>>>> do a lot with an IOMMU. So in essence it's an unsupported feature.
>>
>>
>>>> It's too bad though, it seemed to be a very nice virtualization
>>
>>> platform,
>>
>>>> 6 cores, iommu, affordable ...
>>
>>
>>>> Hmm guess i'm going to write every manufacturer if they supported,
>>
>>> and/or
>>
>>>> if there are plans to support it.
>>
>>>> Hopefully one will do it in the end ...
>>
>>
>>>> --
>>
>>>> Sander
>>
>>
>>>>>> It's a pity that a nice feature isn't supported. No sense in
>>>>>> upgrading
>>
>>>>>> my
>>
>>>>>> setup then to a six core and mobo with IOMMU. :(
>>
>>
>>
>>>>>> --
>>
>>>>>> Sander
>>
>>
>>
>>
>>>>>>> Gigabyte motherboards don't support IOMMU:
>>
>>
>>> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/
>>
>>
>>>>>>> MSI boards have the same problems:
>>
>>>>>>> http://comments.gmane.org/gmane.comp.emulators.xen.user/58630
>>
>>
>>>>>>> ASUS Crosshair IV Formula has IOMMU, but I cannot find if it works
>>
>>> with
>>
>>>>>>> Xen.
>>
>>
>>
>>
>>
>>>>>>> On Sun, May 9, 2010 at 11:38 PM, Sander Eikelenboom
>>
>>>>>>> <linux@eikelenboom.it>wrote:
>>
>>
>>>>>>>> Hello Wei,
>>
>>
>>>>>>>> I hope motherboard manufacturers do a better job with supporting
>>
>>> this
>>
>>>>>>>> iommu
>>
>>>>>>>> in the bios than they did with intel's. Especially Asus made a mess
>>
>>> of
>>
>>>>>>>> it,
>>
>>>>>>>> and hasn't fixed it in newer bioses for multiple motherboards.
>>
>>>>>>>> And since all motherboard reviews don't mention nor test the iommu
>>
>>>>>>>> it's
>>
>>>>>>>> hard to know upfront if it's supported :-(
>>
>>
>>>>>>>> --
>>
>>
>>>>>>>> Sander
>>
>>
>>>>>>>>> Leonardo,
>>
>>
>>>>>>>>> I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly
>>
>>>>>>>> mentioned by external sites, such as
>>
>>>>>>>> http://techreport.com/articles.x/18825. It looks like Asus,
>>>>>>>> Gigabyte
>>
>>>>>>>> and
>>
>>>>>>>> MSI will ship motherboards with 890FX chipset. Xen 4.0 fully
>>
>>> supports
>>
>>>>>>>> AMD
>>
>>>>>>>> IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS
>>
>>> table).
>>
>>>>>>>> Please pay a bit attention to this area.
>>
>>
>>>>>>>>> -Wei
>>
>>
>>>>>>>>> From: xen-devel-bounces@lists.xensource.com [mailto:
>>
>>>>>>>> xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo
>>
>>> Prosperi
>>
>>>>>>>>> Sent: Sunday, May 09, 2010 1:02 PM
>>
>>>>>>>>> To: xen-devel@lists.xensource.com
>>
>>>>>>>>> Subject: [Xen-devel] IOMMU and AMD 890fx
>>
>>
>>
>>>>>>>>> Hi,
>>
>>>>>>>>> All the reviews for the AMD 890fx chipset reported that it
>>
>>> supports
>>
>>>>>>>> IOMMU
>>
>>>>>>>> v1.2.
>>
>>>>>>>>> But I cannot find any mention of that on AMD site.
>>
>>
>>>>>>>>> Does 890fx support IOMMU?
>>
>>>>>>>>> Do 890fx motherboards with IOMMU exist?
>>
>>>>>>>>> Does Xen support 890fx?
>>
>>
>>>>>>>>> Regards,
>>
>>>>>>>>> Leonardo
>>
>>
>>
>>
>>
>>>>>>>> --
>>
>>>>>>>> Best regards,
>>
>>>>>>>> Sander mailto:linux@eikelenboom.it
>>
>>
>>
>>
>>
>>
>>>>>> --
>>
>>>>>> Best regards,
>>
>>>>>> Sander mailto:linux@eikelenboom.it
>>
>>
>>
>>>>>> _______________________________________________
>>
>>>>>> Xen-devel mailing list
>>
>>>>>> Xen-devel@lists.xensource.com
>>
>>>>>> http://lists.xensource.com/xen-devel
>>
>>
>>
>>
>
>

Re: IOMMU and AMD 890fx

[Sander Eikelenboom]

Hi All,

I have send a mail to a number of motherboard manufacturers about the IOMMU support in their bioses.
At the moment MSI has replied (in Dutch) that the MSI 890FXA-GD70 should support it. But it would be nice if it has user confirmation.

I'm  thinking about putting a table on the wiki, stating the replies from tech. supports, and a column with user experiences.


--

Sander


Thursday, June 10, 2010, 9:12:03 AM, you wrote:

> Hi!

> I ordered a Asus Crosshair Formula IV for testing. Will report next week 
> if it works.

> And I contacted Asrock support again and told them that Asus supports 
> it. They will give the informations to their bios-team in Taiwan.

> Meanwhile there is a very good and active discussion in the forum 
> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/index7.html 
> about boards with 890fx and iommu.

> Jens


> Am 09.06.2010 22:26, schrieb Wei Huang:
>> Hi,
>>
>> We talked with AMD client managers. They believe all 890FX platforms
>> should be capable of supporting IOMMU. There must be mis-communication
>> with motherboard vendors. We are in the process of contacting with them.
>> Hopefully the vendors will turn on this feature soon. For now, please
>> just purchase the motherboards with IOMMU enabled (I think Joerg pointed
>> out ASUS at least. I can't verify it myself though.).
>>
>> Best,
>> -Wei
>>
>> Sander Eikelenboom wrote:
>>> So in principle they would like to support it , but they can't / don't
>>> know how.
>>> Very interested at AMD's answer then, unfortunately Wei Huang seems to
>>> be on vacation until June 7th.
>>>
>>> For what i understood in the past was that AMD's solution would not
>>> depend on cpu support (as intel's vt-d does).
>>> And with vga passthrough, iommu on the desktop looks like a good use
>>> of an iommu in my opinion.
>>> --
>>> Sander
>>>
>>>
>>> Thursday, June 3, 2010, 12:51:23 PM, you wrote:
>>>
>>>
>>>> I got this answer from asrock support:
>>>
>>>
>>>
>>>> <answer>
>>>
>>>> So, I get more/final answer from Taiwan department:
>>>
>>>
>>>
>>>> For H/W part:
>>>
>>>> From chipset spec, RD890(890FX) did support IOMMU.
>>>
>>>> But IOMMU also need CPU to support. Till now, AMD has no plan to support
>>>
>>>> IOMMU on desktop CPU (AM3 socket).
>>>
>>>> It only support with server CPU.
>>>
>>>
>>>
>>>> For BIOS part:
>>>
>>>> AMD has removed IOMMU related registers.
>>>
>>>> So there is no chance to let IOMMU work on our 890FX board unless AMD
>>>> can
>>>
>>>> provide such a process, and registers.
>>>
>>>
>>>
>>>> Best regards,
>>>
>>>> ASRock Support
>>>
>>>> </answer>
>>>
>>>
>>>
>>>> So they say it is AMD's fault. :-(
>>>
>>>
>>>
>>>> Regards,
>>>
>>>> Jens
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>> On Thu, 3 Jun 2010 11:23:10 +0200, Sander Eikelenboom
>>>
>>>> <linux@eikelenboom.it> wrote:
>>>
>>>>> Thursday, June 3, 2010, 10:53:29 AM, you wrote:
>>>
>>>
>>>>>> The ASRock 890FX Deluxe3 doesn't support the IOMMU technology too.
>>>
>>>>>> http://forums.amd.com/forum/messageview.cfm?catid=383&threadid=134410
>>>
>>>
>>>>>> I'm very disappointed that main board manufacturers don't enable all
>>>
>>>>>> features for the processor and main board.
>>>
>>>
>>>
>>>>>> On Thu, June 3, 2010 10:19 am, Sander Eikelenboom wrote:
>>>
>>>>>>> Thx for the forum threads, i have changed the wiki about
>>>>>>> vt-d/iommu to
>>>
>>>>>>> reflect these new experiences.
>>>
>>>
>>>>>>> I was hoping AMD would be able to do a better job than Intel, and
>>>>>>> have
>>>
>>>>>>> learned from the BIOS missery Intel has with there VT-d, and
>>>
>>>>>>> instruct/help
>>>
>>>>>>> the manafacturers getting their bioses right.
>>>
>>>
>>>>>> I have a feeling that it's not AMD but the main board manufacturers
>>>>>> who
>>>
>>>>>> don't enable IOMMU in the BIOS. (It might be a hidden feature, for
>>>
>>>> people
>>>
>>>>>> who want to hack the BIOS at your own risk).
>>>
>>>>>> http://www.rebelshavenforum.com/sis-bin/ultimatebb.cgi
>>>
>>>
>>>>> Ofcourse you are right, but as chipset manufacturer I would be pretty
>>>
>>>>> pissed and contact them.
>>>
>>>>> Also I don't know how hard AMD and Intel make it to implement the
>>>
>>>> feature
>>>
>>>>> in the bios. Since most of the time it's or disabled or broken.
>>>
>>>>> On the other hand, they only support "microsoft windows" and that
>>>
>>>> doesn't
>>>
>>>>> do a lot with an IOMMU. So in essence it's an unsupported feature.
>>>
>>>
>>>>> It's too bad though, it seemed to be a very nice virtualization
>>>
>>>> platform,
>>>
>>>>> 6 cores, iommu, affordable ...
>>>
>>>
>>>>> Hmm guess i'm going to write every manufacturer if they supported,
>>>
>>>> and/or
>>>
>>>>> if there are plans to support it.
>>>
>>>>> Hopefully one will do it in the end ...
>>>
>>>
>>>>> --
>>>
>>>>> Sander
>>>
>>>
>>>>>>> It's a pity that a nice feature isn't supported. No sense in
>>>>>>> upgrading
>>>
>>>>>>> my
>>>
>>>>>>> setup then to a six core and mobo with IOMMU. :(
>>>
>>>
>>>
>>>>>>> --
>>>
>>>>>>> Sander
>>>
>>>
>>>
>>>
>>>>>>>> Gigabyte motherboards don't support IOMMU:
>>>
>>>
>>>> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/
>>>
>>>
>>>>>>>> MSI boards have the same problems:
>>>
>>>>>>>> http://comments.gmane.org/gmane.comp.emulators.xen.user/58630
>>>
>>>
>>>>>>>> ASUS Crosshair IV Formula has IOMMU, but I cannot find if it works
>>>
>>>> with
>>>
>>>>>>>> Xen.
>>>
>>>
>>>
>>>
>>>
>>>>>>>> On Sun, May 9, 2010 at 11:38 PM, Sander Eikelenboom
>>>
>>>>>>>> <linux@eikelenboom.it>wrote:
>>>
>>>
>>>>>>>>> Hello Wei,
>>>
>>>
>>>>>>>>> I hope motherboard manufacturers do a better job with supporting
>>>
>>>> this
>>>
>>>>>>>>> iommu
>>>
>>>>>>>>> in the bios than they did with intel's. Especially Asus made a mess
>>>
>>>> of
>>>
>>>>>>>>> it,
>>>
>>>>>>>>> and hasn't fixed it in newer bioses for multiple motherboards.
>>>
>>>>>>>>> And since all motherboard reviews don't mention nor test the iommu
>>>
>>>>>>>>> it's
>>>
>>>>>>>>> hard to know upfront if it's supported :-(
>>>
>>>
>>>>>>>>> --
>>>
>>>
>>>>>>>>> Sander
>>>
>>>
>>>>>>>>>> Leonardo,
>>>
>>>
>>>>>>>>>> I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly
>>>
>>>>>>>>> mentioned by external sites, such as
>>>
>>>>>>>>> http://techreport.com/articles.x/18825. It looks like Asus,
>>>>>>>>> Gigabyte
>>>
>>>>>>>>> and
>>>
>>>>>>>>> MSI will ship motherboards with 890FX chipset. Xen 4.0 fully
>>>
>>>> supports
>>>
>>>>>>>>> AMD
>>>
>>>>>>>>> IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS
>>>
>>>> table).
>>>
>>>>>>>>> Please pay a bit attention to this area.
>>>
>>>
>>>>>>>>>> -Wei
>>>
>>>
>>>>>>>>>> From: xen-devel-bounces@lists.xensource.com [mailto:
>>>
>>>>>>>>> xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo
>>>
>>>> Prosperi
>>>
>>>>>>>>>> Sent: Sunday, May 09, 2010 1:02 PM
>>>
>>>>>>>>>> To: xen-devel@lists.xensource.com
>>>
>>>>>>>>>> Subject: [Xen-devel] IOMMU and AMD 890fx
>>>
>>>
>>>
>>>>>>>>>> Hi,
>>>
>>>>>>>>>> All the reviews for the AMD 890fx chipset reported that it
>>>
>>>> supports
>>>
>>>>>>>>> IOMMU
>>>
>>>>>>>>> v1.2.
>>>
>>>>>>>>>> But I cannot find any mention of that on AMD site.
>>>
>>>
>>>>>>>>>> Does 890fx support IOMMU?
>>>
>>>>>>>>>> Do 890fx motherboards with IOMMU exist?
>>>
>>>>>>>>>> Does Xen support 890fx?
>>>
>>>
>>>>>>>>>> Regards,
>>>
>>>>>>>>>> Leonardo
>>>
>>>
>>>
>>>
>>>
>>>>>>>>> --
>>>
>>>>>>>>> Best regards,
>>>
>>>>>>>>> Sander mailto:linux@eikelenboom.it
>>>
>>>
>>>
>>>
>>>
>>>
>>>>>>> --
>>>
>>>>>>> Best regards,
>>>
>>>>>>> Sander mailto:linux@eikelenboom.it
>>>
>>>
>>>
>>>>>>> _______________________________________________
>>>
>>>>>>> Xen-devel mailing list
>>>
>>>>>>> Xen-devel@lists.xensource.com
>>>
>>>>>>> http://lists.xensource.com/xen-devel
>>>
>>>
>>>
>>>
>>
>>



-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Asus has approved, that all their 890FX-Boards support IOMMU:

http://vip.asus.com/forum/view.aspx?id=20100609185547531&SLanguage=en-us&board_id=1&model=M4A89TD%20PRO/USB3

Jens


Am 10.06.2010 09:23, schrieb Sander Eikelenboom:
> Hi All,
>
> I have send a mail to a number of motherboard manufacturers about the IOMMU support in their bioses.
> At the moment MSI has replied (in Dutch) that the MSI 890FXA-GD70 should support it. But it would be nice if it has user confirmation.
>
> I'm  thinking about putting a table on the wiki, stating the replies from tech. supports, and a column with user experiences.
>
>
> --
>
> Sander
>
>
> Thursday, June 10, 2010, 9:12:03 AM, you wrote:
>
>> Hi!
>
>> I ordered a Asus Crosshair Formula IV for testing. Will report next week
>> if it works.
>
>> And I contacted Asrock support again and told them that Asus supports
>> it. They will give the informations to their bios-team in Taiwan.
>
>> Meanwhile there is a very good and active discussion in the forum
>> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/index7.html
>> about boards with 890fx and iommu.
>
>> Jens
>
>
>> Am 09.06.2010 22:26, schrieb Wei Huang:
>>> Hi,
>>>
>>> We talked with AMD client managers. They believe all 890FX platforms
>>> should be capable of supporting IOMMU. There must be mis-communication
>>> with motherboard vendors. We are in the process of contacting with them.
>>> Hopefully the vendors will turn on this feature soon. For now, please
>>> just purchase the motherboards with IOMMU enabled (I think Joerg pointed
>>> out ASUS at least. I can't verify it myself though.).
>>>
>>> Best,
>>> -Wei
>>>
>>> Sander Eikelenboom wrote:
>>>> So in principle they would like to support it , but they can't / don't
>>>> know how.
>>>> Very interested at AMD's answer then, unfortunately Wei Huang seems to
>>>> be on vacation until June 7th.
>>>>
>>>> For what i understood in the past was that AMD's solution would not
>>>> depend on cpu support (as intel's vt-d does).
>>>> And with vga passthrough, iommu on the desktop looks like a good use
>>>> of an iommu in my opinion.
>>>> --
>>>> Sander
>>>>
>>>>
>>>> Thursday, June 3, 2010, 12:51:23 PM, you wrote:
>>>>
>>>>
>>>>> I got this answer from asrock support:
>>>>
>>>>
>>>>
>>>>> <answer>
>>>>
>>>>> So, I get more/final answer from Taiwan department:
>>>>
>>>>
>>>>
>>>>> For H/W part:
>>>>
>>>>>  From chipset spec, RD890(890FX) did support IOMMU.
>>>>
>>>>> But IOMMU also need CPU to support. Till now, AMD has no plan to support
>>>>
>>>>> IOMMU on desktop CPU (AM3 socket).
>>>>
>>>>> It only support with server CPU.
>>>>
>>>>
>>>>
>>>>> For BIOS part:
>>>>
>>>>> AMD has removed IOMMU related registers.
>>>>
>>>>> So there is no chance to let IOMMU work on our 890FX board unless AMD
>>>>> can
>>>>
>>>>> provide such a process, and registers.
>>>>
>>>>
>>>>
>>>>> Best regards,
>>>>
>>>>> ASRock Support
>>>>
>>>>> </answer>
>>>>
>>>>
>>>>
>>>>> So they say it is AMD's fault. :-(
>>>>
>>>>
>>>>
>>>>> Regards,
>>>>
>>>>> Jens
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> On Thu, 3 Jun 2010 11:23:10 +0200, Sander Eikelenboom
>>>>
>>>>> <linux@eikelenboom.it>  wrote:
>>>>
>>>>>> Thursday, June 3, 2010, 10:53:29 AM, you wrote:
>>>>
>>>>
>>>>>>> The ASRock 890FX Deluxe3 doesn't support the IOMMU technology too.
>>>>
>>>>>>> http://forums.amd.com/forum/messageview.cfm?catid=383&threadid=134410
>>>>
>>>>
>>>>>>> I'm very disappointed that main board manufacturers don't enable all
>>>>
>>>>>>> features for the processor and main board.
>>>>
>>>>
>>>>
>>>>>>> On Thu, June 3, 2010 10:19 am, Sander Eikelenboom wrote:
>>>>
>>>>>>>> Thx for the forum threads, i have changed the wiki about
>>>>>>>> vt-d/iommu to
>>>>
>>>>>>>> reflect these new experiences.
>>>>
>>>>
>>>>>>>> I was hoping AMD would be able to do a better job than Intel, and
>>>>>>>> have
>>>>
>>>>>>>> learned from the BIOS missery Intel has with there VT-d, and
>>>>
>>>>>>>> instruct/help
>>>>
>>>>>>>> the manafacturers getting their bioses right.
>>>>
>>>>
>>>>>>> I have a feeling that it's not AMD but the main board manufacturers
>>>>>>> who
>>>>
>>>>>>> don't enable IOMMU in the BIOS. (It might be a hidden feature, for
>>>>
>>>>> people
>>>>
>>>>>>> who want to hack the BIOS at your own risk).
>>>>
>>>>>>> http://www.rebelshavenforum.com/sis-bin/ultimatebb.cgi
>>>>
>>>>
>>>>>> Ofcourse you are right, but as chipset manufacturer I would be pretty
>>>>
>>>>>> pissed and contact them.
>>>>
>>>>>> Also I don't know how hard AMD and Intel make it to implement the
>>>>
>>>>> feature
>>>>
>>>>>> in the bios. Since most of the time it's or disabled or broken.
>>>>
>>>>>> On the other hand, they only support "microsoft windows" and that
>>>>
>>>>> doesn't
>>>>
>>>>>> do a lot with an IOMMU. So in essence it's an unsupported feature.
>>>>
>>>>
>>>>>> It's too bad though, it seemed to be a very nice virtualization
>>>>
>>>>> platform,
>>>>
>>>>>> 6 cores, iommu, affordable ...
>>>>
>>>>
>>>>>> Hmm guess i'm going to write every manufacturer if they supported,
>>>>
>>>>> and/or
>>>>
>>>>>> if there are plans to support it.
>>>>
>>>>>> Hopefully one will do it in the end ...
>>>>
>>>>
>>>>>> --
>>>>
>>>>>> Sander
>>>>
>>>>
>>>>>>>> It's a pity that a nice feature isn't supported. No sense in
>>>>>>>> upgrading
>>>>
>>>>>>>> my
>>>>
>>>>>>>> setup then to a six core and mobo with IOMMU. :(
>>>>
>>>>
>>>>
>>>>>>>> --
>>>>
>>>>>>>> Sander
>>>>
>>>>
>>>>
>>>>
>>>>>>>>> Gigabyte motherboards don't support IOMMU:
>>>>
>>>>
>>>>> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/
>>>>
>>>>
>>>>>>>>> MSI boards have the same problems:
>>>>
>>>>>>>>> http://comments.gmane.org/gmane.comp.emulators.xen.user/58630
>>>>
>>>>
>>>>>>>>> ASUS Crosshair IV Formula has IOMMU, but I cannot find if it works
>>>>
>>>>> with
>>>>
>>>>>>>>> Xen.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>>>>> On Sun, May 9, 2010 at 11:38 PM, Sander Eikelenboom
>>>>
>>>>>>>>> <linux@eikelenboom.it>wrote:
>>>>
>>>>
>>>>>>>>>> Hello Wei,
>>>>
>>>>
>>>>>>>>>> I hope motherboard manufacturers do a better job with supporting
>>>>
>>>>> this
>>>>
>>>>>>>>>> iommu
>>>>
>>>>>>>>>> in the bios than they did with intel's. Especially Asus made a mess
>>>>
>>>>> of
>>>>
>>>>>>>>>> it,
>>>>
>>>>>>>>>> and hasn't fixed it in newer bioses for multiple motherboards.
>>>>
>>>>>>>>>> And since all motherboard reviews don't mention nor test the iommu
>>>>
>>>>>>>>>> it's
>>>>
>>>>>>>>>> hard to know upfront if it's supported :-(
>>>>
>>>>
>>>>>>>>>> --
>>>>
>>>>
>>>>>>>>>> Sander
>>>>
>>>>
>>>>>>>>>>> Leonardo,
>>>>
>>>>
>>>>>>>>>>> I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly
>>>>
>>>>>>>>>> mentioned by external sites, such as
>>>>
>>>>>>>>>> http://techreport.com/articles.x/18825. It looks like Asus,
>>>>>>>>>> Gigabyte
>>>>
>>>>>>>>>> and
>>>>
>>>>>>>>>> MSI will ship motherboards with 890FX chipset. Xen 4.0 fully
>>>>
>>>>> supports
>>>>
>>>>>>>>>> AMD
>>>>
>>>>>>>>>> IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS
>>>>
>>>>> table).
>>>>
>>>>>>>>>> Please pay a bit attention to this area.
>>>>
>>>>
>>>>>>>>>>> -Wei
>>>>
>>>>
>>>>>>>>>>> From: xen-devel-bounces@lists.xensource.com [mailto:
>>>>
>>>>>>>>>> xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo
>>>>
>>>>> Prosperi
>>>>
>>>>>>>>>>> Sent: Sunday, May 09, 2010 1:02 PM
>>>>
>>>>>>>>>>> To: xen-devel@lists.xensource.com
>>>>
>>>>>>>>>>> Subject: [Xen-devel] IOMMU and AMD 890fx
>>>>
>>>>
>>>>
>>>>>>>>>>> Hi,
>>>>
>>>>>>>>>>> All the reviews for the AMD 890fx chipset reported that it
>>>>
>>>>> supports
>>>>
>>>>>>>>>> IOMMU
>>>>
>>>>>>>>>> v1.2.
>>>>
>>>>>>>>>>> But I cannot find any mention of that on AMD site.
>>>>
>>>>
>>>>>>>>>>> Does 890fx support IOMMU?
>>>>
>>>>>>>>>>> Do 890fx motherboards with IOMMU exist?
>>>>
>>>>>>>>>>> Does Xen support 890fx?
>>>>
>>>>
>>>>>>>>>>> Regards,
>>>>
>>>>>>>>>>> Leonardo
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>>>>>> --
>>>>
>>>>>>>>>> Best regards,
>>>>
>>>>>>>>>> Sander mailto:linux@eikelenboom.it
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>>>> --
>>>>
>>>>>>>> Best regards,
>>>>
>>>>>>>> Sander mailto:linux@eikelenboom.it
>>>>
>>>>
>>>>
>>>>>>>> _______________________________________________
>>>>
>>>>>>>> Xen-devel mailing list
>>>>
>>>>>>>> Xen-devel@lists.xensource.com
>>>>
>>>>>>>> http://lists.xensource.com/xen-devel
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>
>
>

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Hi all!

I got this answer in a forum about this MSI board:

...
I have this board (MSI 890FXA-GD70), there is no bios support for IOMMU. 
MSI presales told me that it was supported before I bought it, I am 
thinking there might have been some miscommunication there. I have been 
waiting about a week for a response as to when/if they plan on adding 
support in the bios.
...


Jens

Am 10.06.2010 09:23, schrieb Sander Eikelenboom:
> Hi All,
>
> I have send a mail to a number of motherboard manufacturers about the IOMMU support in their bioses.
> At the moment MSI has replied (in Dutch) that the MSI 890FXA-GD70 should support it. But it would be nice if it has user confirmation.
>
> I'm  thinking about putting a table on the wiki, stating the replies from tech. supports, and a column with user experiences.
>
>
> --
>
> Sander
>
>
> Thursday, June 10, 2010, 9:12:03 AM, you wrote:
>
>> Hi!
>
>> I ordered a Asus Crosshair Formula IV for testing. Will report next week
>> if it works.
>
>> And I contacted Asrock support again and told them that Asus supports
>> it. They will give the informations to their bios-team in Taiwan.
>
>> Meanwhile there is a very good and active discussion in the forum
>> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/index7.html
>> about boards with 890fx and iommu.
>
>> Jens
>
>
>> Am 09.06.2010 22:26, schrieb Wei Huang:
>>> Hi,
>>>
>>> We talked with AMD client managers. They believe all 890FX platforms
>>> should be capable of supporting IOMMU. There must be mis-communication
>>> with motherboard vendors. We are in the process of contacting with them.
>>> Hopefully the vendors will turn on this feature soon. For now, please
>>> just purchase the motherboards with IOMMU enabled (I think Joerg pointed
>>> out ASUS at least. I can't verify it myself though.).
>>>
>>> Best,
>>> -Wei
>>>
>>> Sander Eikelenboom wrote:
>>>> So in principle they would like to support it , but they can't / don't
>>>> know how.
>>>> Very interested at AMD's answer then, unfortunately Wei Huang seems to
>>>> be on vacation until June 7th.
>>>>
>>>> For what i understood in the past was that AMD's solution would not
>>>> depend on cpu support (as intel's vt-d does).
>>>> And with vga passthrough, iommu on the desktop looks like a good use
>>>> of an iommu in my opinion.
>>>> --
>>>> Sander
>>>>
>>>>
>>>> Thursday, June 3, 2010, 12:51:23 PM, you wrote:
>>>>
>>>>
>>>>> I got this answer from asrock support:
>>>>
>>>>
>>>>
>>>>> <answer>
>>>>
>>>>> So, I get more/final answer from Taiwan department:
>>>>
>>>>
>>>>
>>>>> For H/W part:
>>>>
>>>>>  From chipset spec, RD890(890FX) did support IOMMU.
>>>>
>>>>> But IOMMU also need CPU to support. Till now, AMD has no plan to support
>>>>
>>>>> IOMMU on desktop CPU (AM3 socket).
>>>>
>>>>> It only support with server CPU.
>>>>
>>>>
>>>>
>>>>> For BIOS part:
>>>>
>>>>> AMD has removed IOMMU related registers.
>>>>
>>>>> So there is no chance to let IOMMU work on our 890FX board unless AMD
>>>>> can
>>>>
>>>>> provide such a process, and registers.
>>>>
>>>>
>>>>
>>>>> Best regards,
>>>>
>>>>> ASRock Support
>>>>
>>>>> </answer>
>>>>
>>>>
>>>>
>>>>> So they say it is AMD's fault. :-(
>>>>
>>>>
>>>>
>>>>> Regards,
>>>>
>>>>> Jens
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> On Thu, 3 Jun 2010 11:23:10 +0200, Sander Eikelenboom
>>>>
>>>>> <linux@eikelenboom.it>  wrote:
>>>>
>>>>>> Thursday, June 3, 2010, 10:53:29 AM, you wrote:
>>>>
>>>>
>>>>>>> The ASRock 890FX Deluxe3 doesn't support the IOMMU technology too.
>>>>
>>>>>>> http://forums.amd.com/forum/messageview.cfm?catid=383&threadid=134410
>>>>
>>>>
>>>>>>> I'm very disappointed that main board manufacturers don't enable all
>>>>
>>>>>>> features for the processor and main board.
>>>>
>>>>
>>>>
>>>>>>> On Thu, June 3, 2010 10:19 am, Sander Eikelenboom wrote:
>>>>
>>>>>>>> Thx for the forum threads, i have changed the wiki about
>>>>>>>> vt-d/iommu to
>>>>
>>>>>>>> reflect these new experiences.
>>>>
>>>>
>>>>>>>> I was hoping AMD would be able to do a better job than Intel, and
>>>>>>>> have
>>>>
>>>>>>>> learned from the BIOS missery Intel has with there VT-d, and
>>>>
>>>>>>>> instruct/help
>>>>
>>>>>>>> the manafacturers getting their bioses right.
>>>>
>>>>
>>>>>>> I have a feeling that it's not AMD but the main board manufacturers
>>>>>>> who
>>>>
>>>>>>> don't enable IOMMU in the BIOS. (It might be a hidden feature, for
>>>>
>>>>> people
>>>>
>>>>>>> who want to hack the BIOS at your own risk).
>>>>
>>>>>>> http://www.rebelshavenforum.com/sis-bin/ultimatebb.cgi
>>>>
>>>>
>>>>>> Ofcourse you are right, but as chipset manufacturer I would be pretty
>>>>
>>>>>> pissed and contact them.
>>>>
>>>>>> Also I don't know how hard AMD and Intel make it to implement the
>>>>
>>>>> feature
>>>>
>>>>>> in the bios. Since most of the time it's or disabled or broken.
>>>>
>>>>>> On the other hand, they only support "microsoft windows" and that
>>>>
>>>>> doesn't
>>>>
>>>>>> do a lot with an IOMMU. So in essence it's an unsupported feature.
>>>>
>>>>
>>>>>> It's too bad though, it seemed to be a very nice virtualization
>>>>
>>>>> platform,
>>>>
>>>>>> 6 cores, iommu, affordable ...
>>>>
>>>>
>>>>>> Hmm guess i'm going to write every manufacturer if they supported,
>>>>
>>>>> and/or
>>>>
>>>>>> if there are plans to support it.
>>>>
>>>>>> Hopefully one will do it in the end ...
>>>>
>>>>
>>>>>> --
>>>>
>>>>>> Sander
>>>>
>>>>
>>>>>>>> It's a pity that a nice feature isn't supported. No sense in
>>>>>>>> upgrading
>>>>
>>>>>>>> my
>>>>
>>>>>>>> setup then to a six core and mobo with IOMMU. :(
>>>>
>>>>
>>>>
>>>>>>>> --
>>>>
>>>>>>>> Sander
>>>>
>>>>
>>>>
>>>>
>>>>>>>>> Gigabyte motherboards don't support IOMMU:
>>>>
>>>>
>>>>> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/
>>>>
>>>>
>>>>>>>>> MSI boards have the same problems:
>>>>
>>>>>>>>> http://comments.gmane.org/gmane.comp.emulators.xen.user/58630
>>>>
>>>>
>>>>>>>>> ASUS Crosshair IV Formula has IOMMU, but I cannot find if it works
>>>>
>>>>> with
>>>>
>>>>>>>>> Xen.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>>>>> On Sun, May 9, 2010 at 11:38 PM, Sander Eikelenboom
>>>>
>>>>>>>>> <linux@eikelenboom.it>wrote:
>>>>
>>>>
>>>>>>>>>> Hello Wei,
>>>>
>>>>
>>>>>>>>>> I hope motherboard manufacturers do a better job with supporting
>>>>
>>>>> this
>>>>
>>>>>>>>>> iommu
>>>>
>>>>>>>>>> in the bios than they did with intel's. Especially Asus made a mess
>>>>
>>>>> of
>>>>
>>>>>>>>>> it,
>>>>
>>>>>>>>>> and hasn't fixed it in newer bioses for multiple motherboards.
>>>>
>>>>>>>>>> And since all motherboard reviews don't mention nor test the iommu
>>>>
>>>>>>>>>> it's
>>>>
>>>>>>>>>> hard to know upfront if it's supported :-(
>>>>
>>>>
>>>>>>>>>> --
>>>>
>>>>
>>>>>>>>>> Sander
>>>>
>>>>
>>>>>>>>>>> Leonardo,
>>>>
>>>>
>>>>>>>>>>> I can confirm that AMD 890FX supports IOMMU v1.22. It was briefly
>>>>
>>>>>>>>>> mentioned by external sites, such as
>>>>
>>>>>>>>>> http://techreport.com/articles.x/18825. It looks like Asus,
>>>>>>>>>> Gigabyte
>>>>
>>>>>>>>>> and
>>>>
>>>>>>>>>> MSI will ship motherboards with 890FX chipset. Xen 4.0 fully
>>>>
>>>>> supports
>>>>
>>>>>>>>>> AMD
>>>>
>>>>>>>>>> IOMMU v1.2. Note that IOMMU needs support from BIOS (ACPI IVRS
>>>>
>>>>> table).
>>>>
>>>>>>>>>> Please pay a bit attention to this area.
>>>>
>>>>
>>>>>>>>>>> -Wei
>>>>
>>>>
>>>>>>>>>>> From: xen-devel-bounces@lists.xensource.com [mailto:
>>>>
>>>>>>>>>> xen-devel-bounces@lists.xensource.com] On Behalf Of Leonardo
>>>>
>>>>> Prosperi
>>>>
>>>>>>>>>>> Sent: Sunday, May 09, 2010 1:02 PM
>>>>
>>>>>>>>>>> To: xen-devel@lists.xensource.com
>>>>
>>>>>>>>>>> Subject: [Xen-devel] IOMMU and AMD 890fx
>>>>
>>>>
>>>>
>>>>>>>>>>> Hi,
>>>>
>>>>>>>>>>> All the reviews for the AMD 890fx chipset reported that it
>>>>
>>>>> supports
>>>>
>>>>>>>>>> IOMMU
>>>>
>>>>>>>>>> v1.2.
>>>>
>>>>>>>>>>> But I cannot find any mention of that on AMD site.
>>>>
>>>>
>>>>>>>>>>> Does 890fx support IOMMU?
>>>>
>>>>>>>>>>> Do 890fx motherboards with IOMMU exist?
>>>>
>>>>>>>>>>> Does Xen support 890fx?
>>>>
>>>>
>>>>>>>>>>> Regards,
>>>>
>>>>>>>>>>> Leonardo
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>>>>>> --
>>>>
>>>>>>>>>> Best regards,
>>>>
>>>>>>>>>> Sander mailto:linux@eikelenboom.it
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>>>> --
>>>>
>>>>>>>> Best regards,
>>>>
>>>>>>>> Sander mailto:linux@eikelenboom.it
>>>>
>>>>
>>>>
>>>>>>>> _______________________________________________
>>>>
>>>>>>>> Xen-devel mailing list
>>>>
>>>>>>>> Xen-devel@lists.xensource.com
>>>>
>>>>>>>> http://lists.xensource.com/xen-devel
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>
>
>

Re: IOMMU and AMD 890fx

[Andrew Oakley]

On Thu, 3 Jun 2010 13:51:05 +0200
Andre Przywara <andre.przywara@amd.com> wrote:
> If the ACPI table is missing, you can try to look for the
> corresponding PCI device: it should have a vendor/device-id of
> 1002:5a23, lspci lists it as:
> 00:00.2 Generic system peripheral [0806]: ATI Technologies Inc Device
> 5a23 If anyone has a board with the PCI device but a missing ACPI
> table, please tell us.
> 
> We are already trying to improve the situation, you could help us by 
> reporting the IOMMU support status on 890FX boards. Simply check for
> the ACPI table:
> # xm dmesg | grep IVRS    or
> $ ls /sys/firmware/acpi/tables/IVRS
> and by checking for the PCI device:
> $ /sbin/lspci -d 1002:5a23

I have a Gigabyte GA-890FXA-UD5 motherboard.  I have opened a support
ticket with Gigabyte but have not yet received any reply.  The
virtualization option in the BIOS is turned on, there do not appear to
be any other relevant options.  

$ lspci -d 1002:5a23
$ ls /sys/firmware/acpi/tables/
APIC  DSDT  dynamic  FACP  FACS  HPET  MATS  MCFG  SSDT  TAMG

If there is any response from Gigabyte to AMD can it be published on
the list please?

Thanks

Re: IOMMU and AMD 890fx

[Wei Huang]

Hi Andrew,

Could you do us a favor by taking a look at chipset options (or 
something similar)? The IOMMU option might be hidden under that menu. 
Virtualization is for enabling CPU's SVM feature. So it is unrelated.

Thanks,
-Wei

Andrew Oakley wrote:
> On Thu, 3 Jun 2010 13:51:05 +0200
> Andre Przywara <andre.przywara@amd.com> wrote:
>   
>> If the ACPI table is missing, you can try to look for the
>> corresponding PCI device: it should have a vendor/device-id of
>> 1002:5a23, lspci lists it as:
>> 00:00.2 Generic system peripheral [0806]: ATI Technologies Inc Device
>> 5a23 If anyone has a board with the PCI device but a missing ACPI
>> table, please tell us.
>>
>> We are already trying to improve the situation, you could help us by 
>> reporting the IOMMU support status on 890FX boards. Simply check for
>> the ACPI table:
>> # xm dmesg | grep IVRS    or
>> $ ls /sys/firmware/acpi/tables/IVRS
>> and by checking for the PCI device:
>> $ /sbin/lspci -d 1002:5a23
>>     
>
> I have a Gigabyte GA-890FXA-UD5 motherboard.  I have opened a support
> ticket with Gigabyte but have not yet received any reply.  The
> virtualization option in the BIOS is turned on, there do not appear to
> be any other relevant options.  
>
> $ lspci -d 1002:5a23
> $ ls /sys/firmware/acpi/tables/
> APIC  DSDT  dynamic  FACP  FACS  HPET  MATS  MCFG  SSDT  TAMG
>
> If there is any response from Gigabyte to AMD can it be published on
> the list please?
>
> Thanks
>
>   

Re: IOMMU and AMD 890fx

[Andrew Oakley]

On Thu, 10 Jun 2010 14:12:17 -0500
Wei Huang <wei.huang2@amd.com> wrote:

> Hi Andrew,
> 
> Could you do us a favor by taking a look at chipset options (or 
> something similar)? The IOMMU option might be hidden under that menu. 
> Virtualization is for enabling CPU's SVM feature. So it is unrelated.

I've been through (all) the BIOS options (again) and I'm sure there are
no options for the IOMMU.  "Virtualization" does seem to turn on SVM.  

> Thanks,
> -Wei
> 
> Andrew Oakley wrote:
> > On Thu, 3 Jun 2010 13:51:05 +0200
> > Andre Przywara <andre.przywara@amd.com> wrote:
> >   
> >> If the ACPI table is missing, you can try to look for the
> >> corresponding PCI device: it should have a vendor/device-id of
> >> 1002:5a23, lspci lists it as:
> >> 00:00.2 Generic system peripheral [0806]: ATI Technologies Inc
> >> Device 5a23 If anyone has a board with the PCI device but a
> >> missing ACPI table, please tell us.
> >>
> >> We are already trying to improve the situation, you could help us
> >> by reporting the IOMMU support status on 890FX boards. Simply
> >> check for the ACPI table:
> >> # xm dmesg | grep IVRS    or
> >> $ ls /sys/firmware/acpi/tables/IVRS
> >> and by checking for the PCI device:
> >> $ /sbin/lspci -d 1002:5a23
> >>     
> >
> > I have a Gigabyte GA-890FXA-UD5 motherboard.  I have opened a
> > support ticket with Gigabyte but have not yet received any reply.
> > The virtualization option in the BIOS is turned on, there do not
> > appear to be any other relevant options.  
> >
> > $ lspci -d 1002:5a23
> > $ ls /sys/firmware/acpi/tables/
> > APIC  DSDT  dynamic  FACP  FACS  HPET  MATS  MCFG  SSDT  TAMG
> >
> > If there is any response from Gigabyte to AMD can it be published on
> > the list please?
> >
> > Thanks
> >
> >   
> 
> 

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Hi!

I now got a Asus Crosshair IV mainboard and here are the outputs after I 
enabled the IOMMU option in the bios:

xen-01:~# ls /sys/firmware/acpi/tables/
APIC  DSDT  dynamic  FACP  FACS  HPET  IVRS  MCFG  OEMB  SRAT  SSDT
xen-01:~# dmesg |grep -i iommu
[    1.691432] AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
xen-01:~# lspci -d 1002:5a23
00:00.2 Generic system peripheral [0806]: ATI Technologies Inc Device 5a23
xen-01:~#

So here is the IOMMU of the 890FX. Will go further for testing now..


Jens


Am 10.06.2010 21:15, schrieb Andrew Oakley:
> On Thu, 10 Jun 2010 14:12:17 -0500
> Wei Huang<wei.huang2@amd.com>  wrote:
>
>> Hi Andrew,
>>
>> Could you do us a favor by taking a look at chipset options (or
>> something similar)? The IOMMU option might be hidden under that menu.
>> Virtualization is for enabling CPU's SVM feature. So it is unrelated.
>
> I've been through (all) the BIOS options (again) and I'm sure there are
> no options for the IOMMU.  "Virtualization" does seem to turn on SVM.
>
>> Thanks,
>> -Wei
>>
>> Andrew Oakley wrote:
>>> On Thu, 3 Jun 2010 13:51:05 +0200
>>> Andre Przywara<andre.przywara@amd.com>  wrote:
>>>
>>>> If the ACPI table is missing, you can try to look for the
>>>> corresponding PCI device: it should have a vendor/device-id of
>>>> 1002:5a23, lspci lists it as:
>>>> 00:00.2 Generic system peripheral [0806]: ATI Technologies Inc
>>>> Device 5a23 If anyone has a board with the PCI device but a
>>>> missing ACPI table, please tell us.
>>>>
>>>> We are already trying to improve the situation, you could help us
>>>> by reporting the IOMMU support status on 890FX boards. Simply
>>>> check for the ACPI table:
>>>> # xm dmesg | grep IVRS    or
>>>> $ ls /sys/firmware/acpi/tables/IVRS
>>>> and by checking for the PCI device:
>>>> $ /sbin/lspci -d 1002:5a23
>>>>
>>>
>>> I have a Gigabyte GA-890FXA-UD5 motherboard.  I have opened a
>>> support ticket with Gigabyte but have not yet received any reply.
>>> The virtualization option in the BIOS is turned on, there do not
>>> appear to be any other relevant options.
>>>
>>> $ lspci -d 1002:5a23
>>> $ ls /sys/firmware/acpi/tables/
>>> APIC  DSDT  dynamic  FACP  FACS  HPET  MATS  MCFG  SSDT  TAMG
>>>
>>> If there is any response from Gigabyte to AMD can it be published on
>>> the list please?
>>>
>>> Thanks
>>>
>>>
>>
>>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: IOMMU and AMD 890fx

[Joerg Roedel]

Hi Jens,

On Mon, Jun 14, 2010 at 01:08:47PM +0200, Jens Krehbiel-Gräther wrote:
> I now got a Asus Crosshair IV mainboard and here are the outputs after I  
> enabled the IOMMU option in the bios:
>
> xen-01:~# ls /sys/firmware/acpi/tables/
> APIC  DSDT  dynamic  FACP  FACS  HPET  IVRS  MCFG  OEMB  SRAT  SSDT
> xen-01:~# dmesg |grep -i iommu
> [    1.691432] AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
> xen-01:~# lspci -d 1002:5a23
> 00:00.2 Generic system peripheral [0806]: ATI Technologies Inc Device 5a23

This are good news. At least one of the numerous 890FX mainboards
support the IOMMU of the chipset in its BIOS. Thanks for trying this
out.

	Joerg

Re: IOMMU and AMD 890fx

[Andre Przywara]

Jens Krehbiel-Gräther wrote:
> Hi!
> 
> I now got a Asus Crosshair IV mainboard and here are the outputs after I 
> enabled the IOMMU option in the bios:
> 
> xen-01:~# ls /sys/firmware/acpi/tables/
> APIC  DSDT  dynamic  FACP  FACS  HPET  IVRS  MCFG  OEMB  SRAT  SSDT
> xen-01:~# dmesg |grep -i iommu
> [    1.691432] AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
> xen-01:~# lspci -d 1002:5a23
> 00:00.2 Generic system peripheral [0806]: ATI Technologies Inc Device 5a23
> xen-01:~#
> 
> So here is the IOMMU of the 890FX. Will go further for testing now..

Great news, thanks for pursuing that.

For the other boards:
We have identified an issue and are working on a solution. Stay tuned.

Also please note that sometimes board vendors also call the AMD64 GART 
an IOMMU, so if you have correspondence with them, make sure you are 
talking about the same thing.

Regards,
Andre.

-- 
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448 3567 12
----to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

Re: IOMMU and AMD 890fx

[Joop Boonen]

I've got the following answer from Biostar in respect to the Biostar
TA890FXE main board.
http://www.biostar.com.tw/app/en/mb/introduction.php?S_ID=482

<quote>
The motherboard supports IOMMU, but the CPU you are installing also must
support IOMMU.



support@biostar-usa.com
626-581-1055 menu select 2 
</quote>


I've just took the risk to order this mainboard as the main board is
relatively cheap and I hope/presume the IOMMU will be supported
eventually if it's not supported yet.
I'll update you when I know the result.
I understood IOMMU is not related to the CPU i.e. IOMMU should also be
supported when the CPU on the main board is a AMD Phenom II X2 550 Black
Edition?

On 06/14/2010 03:49 PM, Andre Przywara wrote:
> Jens Krehbiel-Gräther wrote:
>> Hi!
>>
>> I now got a Asus Crosshair IV mainboard and here are the outputs
>> after I enabled the IOMMU option in the bios:
>>
>> xen-01:~# ls /sys/firmware/acpi/tables/
>> APIC  DSDT  dynamic  FACP  FACS  HPET  IVRS  MCFG  OEMB  SRAT  SSDT
>> xen-01:~# dmesg |grep -i iommu
>> [    1.691432] AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
>> xen-01:~# lspci -d 1002:5a23
>> 00:00.2 Generic system peripheral [0806]: ATI Technologies Inc Device
>> 5a23
>> xen-01:~#
>>
>> So here is the IOMMU of the 890FX. Will go further for testing now..
>
> Great news, thanks for pursuing that.
>
> For the other boards:
> We have identified an issue and are working on a solution. Stay tuned.
>
> Also please note that sometimes board vendors also call the AMD64 GART
> an IOMMU, so if you have correspondence with them, make sure you are
> talking about the same thing.
>
> Regards,
> Andre.
>

RE: IOMMU and AMD 890fx

[Huang2, Wei]

IOMMU doesn't require CPU support. It is a feature of chipset. I think Biostart means SVM feature of CPU, which is supported by Phenom II.


-Wei
________________________________________
From: Joop Boonen [joop_boonen@web.de]
Sent: Tuesday, June 15, 2010 4:37 PM
To: Przywara, Andre
Cc: Jens Krehbiel-Gräther; Leonardo Prosperi; xen-devel@lists.xensource.com; Joerg Roedel; Huang2, Wei; Eikelenboom; Andrew Oakley
Subject: Re: [Xen-devel] IOMMU and AMD 890fx

I've got the following answer from Biostar in respect to the Biostar
TA890FXE main board.
http://www.biostar.com.tw/app/en/mb/introduction.php?S_ID=482

<quote>
The motherboard supports IOMMU, but the CPU you are installing also must
support IOMMU.



support@biostar-usa.com
626-581-1055 menu select 2
</quote>


I've just took the risk to order this mainboard as the main board is
relatively cheap and I hope/presume the IOMMU will be supported
eventually if it's not supported yet.
I'll update you when I know the result.
I understood IOMMU is not related to the CPU i.e. IOMMU should also be
supported when the CPU on the main board is a AMD Phenom II X2 550 Black
Edition?

On 06/14/2010 03:49 PM, Andre Przywara wrote:
> Jens Krehbiel-Gräther wrote:
>> Hi!
>>
>> I now got a Asus Crosshair IV mainboard and here are the outputs
>> after I enabled the IOMMU option in the bios:
>>
>> xen-01:~# ls /sys/firmware/acpi/tables/
>> APIC  DSDT  dynamic  FACP  FACS  HPET  IVRS  MCFG  OEMB  SRAT  SSDT
>> xen-01:~# dmesg |grep -i iommu
>> [    1.691432] AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
>> xen-01:~# lspci -d 1002:5a23
>> 00:00.2 Generic system peripheral [0806]: ATI Technologies Inc Device
>> 5a23
>> xen-01:~#
>>
>> So here is the IOMMU of the 890FX. Will go further for testing now..
>
> Great news, thanks for pursuing that.
>
> For the other boards:
> We have identified an issue and are working on a solution. Stay tuned.
>
> Also please note that sometimes board vendors also call the AMD64 GART
> an IOMMU, so if you have correspondence with them, make sure you are
> talking about the same thing.
>
> Regards,
> Andre.
>

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Hi!

I now have a Asus Crosshair IV for testing and this board (as I postet 
earlier) supports iommu. Now I have installed Debian lenny (5.0) and 
compiled xen 4.0.0.
Xen is running now and I have installed a hvm debian lenny to which I 
would like to bind a pci device but that doesn't work.
Perhaps I am doing something wrong and you could help me?


xen-01:~# xm dmesg |grep -i -e iommu -e amd-vi -e ivrs
(XEN) ACPI: IVRS CFD9FA00, 00C8 (r1  AMD     RD890S   202031 AMD         0)
(XEN) AMD-Vi: IOMMU 0 Enabled.
xen-01:~#

So as xm dmesg reports IOMMU is enabled. I passed the "iommu=1" 
parameter to kernel.

xen-01:~# uname -a
Linux xen-01 2.6.32-bpo.5-xen-amd64 #1 SMP Fri Jun 11 12:21:09 UTC 2010 
x86_64 GNU/Linux

xen-01:/boot# grep -i iommu config-2.6.32-bpo.5-xen-amd64
CONFIG_GART_IOMMU=y
CONFIG_CALGARY_IOMMU=y
CONFIG_CALGARY_IOMMU_ENABLED_BY_DEFAULT=y
CONFIG_AMD_IOMMU=y
# CONFIG_AMD_IOMMU_STATS is not set
CONFIG_IOMMU_HELPER=y
CONFIG_IOMMU_API=y
# CONFIG_IOMMU_DEBUG is not set
# CONFIG_IOMMU_STRESS is not set

xen-01:/boot# grep -i backend config-2.6.32-bpo.5-xen-amd64
CONFIG_XEN_BACKEND=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_PCIDEV_BACKEND=y
CONFIG_XEN_PCIDEV_BACKEND_VPCI=y
# CONFIG_XEN_PCIDEV_BACKEND_PASS is not set
# CONFIG_XEN_PCIDEV_BACKEND_SLOT is not set
# CONFIG_XEN_PCIDEV_BACKEND_CONTROLLER is not set


So I think the needed kernel options are enabled.

I can hide the pci device from dom0:

lspci:
00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) (rev 40)

echo -n "0000:00:14.2" > "/sys/bus/pci/drivers/HDA Intel/unbind"
echo -n "0000:00:14.2" > /sys/bus/pci/drivers/pciback/new_slot
echo -n "0000:00:14.2" > /sys/bus/pci/drivers/pciback/bind


xen-01:~# xm pci-list-assignable-devices
0000:00:14.2


Now when I try to pass through the device to the guest I get (at first 
try) this message:

xen-01:~# xm pci-attach test 0000:00:14.2 8
Error: Timed out waiting for device model action
Usage: xm pci-attach [-o|--options=<opt>] <Domain> 
<domain:bus:slot.func> [virtual slot]

Insert a new pass-through pci device.

When I try it again, I will always get this message:

xen-01:~# xm pci-attach test 0000:00:14.2 8
Error: Failed to assign device to IOMMU 
(0000:00:14.2@40,msitranslate=1,power_mgmt=0)
Usage: xm pci-attach [-o|--options=<opt>] <Domain> 
<domain:bus:slot.func> [virtual slot]

Insert a new pass-through pci device.

In /var/log/xen/xend.log there are this messages:

[2010-06-17 00:06:18 2407] DEBUG (XendDomainInfo:903) 
XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x14'], 
['domain', '0x0000'], ['key', '0000:00:14.2@8'], ['bus', '0x00'], 
['vdevfn', '0x40'], ['func', '0x2']], ['state', 'Initialising']]

[2010-06-17 00:06:18 2407] DEBUG (XendDomainInfo:675) 
XendDomainInfo.hvm_pci_device_create: {'devs': [{'slot': '0x14', 
'domain': '0x0000', 'key': '0000:00:14.2@8', 'bus': '0x00', 'vdevfn': 
'0x40', 'func': '0x2', 'uuid': 'f9a78b41-aad2-8c31-73a3-2b2162c8ebb4'}], 
'states': ['Initialising']}

[2010-06-17 00:06:18 2407] DEBUG (XendDomainInfo:801) 
XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x14', 'domain': 
'0x0000', 'key': '0000:00:14.2@8', 'bus': '0x00', 'vdevfn': '0x40', 
'func': '0x2', 'uuid': 'f9a78b41-aad2-8c31-73a3-2b2162c8ebb4'}

[2010-06-17 00:06:18 2407] DEBUG (XendDomainInfo:822) 
XendDomainInfo.hvm_pci_device_insert_dev: 
0000:00:14.2@40,msitranslate=1,power_mgmt=0


dmesg gives this log:

[ 2177.373375] pciback 0000:00:14.2: seizing device
[ 2177.373488] xen: registering gsi 16 triggering 0 polarity 1
[ 2177.373497] xen_allocate_pirq: returning irq 16 for gsi 16
[ 2177.373519] xen: --> irq=16
[ 2177.373528] Already setup the GSI :16
[ 2177.373551] pciback 0000:00:14.2: PCI INT A -> GSI 16 (level, low) -> 
IRQ 16
[ 2177.373582] pciback 0000:00:14.2: PCI INT A disabled


and xm dmesg this:

(XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device (0:14.2) 
failed


So can anyone tell me what I am doing wrong??

Hope someone could help me!

    Jens

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Am 16.06.2010 00:40, schrieb Huang2, Wei:
> IOMMU doesn't require CPU support. It is a feature of chipset. I think Biostart means SVM feature of CPU, which is supported by Phenom II.
>
>
> -Wei

Perhaps you should tell this asrock. Today I got an answer from asrock 
support:

<asrock-support>
Sorry for lat reply. Taiwan was still checking this issue. And they told us:

Again IOMMU also need CPU to support.
Till now, AMD has no plan to support IOMMU on desktop CPU (AM3 socket).
Also "IOMMU" not list on our 890FX Deluxe3 specification
Please 890FX Deluxe3 link:
http://www.asrock.com/mb/overview.asp?Model=890FX%20Deluxe3

In feature, AMD release that CPU, we will borrow it, and test it.
Any news will update to you.
Thanks your mail.

Best regards,


ASRock Support
</asrock-support>


Jens

Re: IOMMU and AMD 890fx

[Konrad Rzeszutek Wilk]

On Thu, Jun 17, 2010 at 12:09:31AM +0200, Jens Krehbiel-Gräther wrote:
> Hi!
> 
> I now have a Asus Crosshair IV for testing and this board (as I
> postet earlier) supports iommu. Now I have installed Debian lenny
> (5.0) and compiled xen 4.0.0.
> Xen is running now and I have installed a hvm debian lenny to which
> I would like to bind a pci device but that doesn't work.
> Perhaps I am doing something wrong and you could help me?

Is that the _only_ PCI device you have tried to assign? Did you try to
assign other PCI devices which have the function number being zero?
> and xm dmesg this:
> 
> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device
> (0:14.2) failed

That is not very useful. Wish it gave you the return code at least.
If you are comfortable with adding printks in the Xen hypervisor I would 
modify it a bit to see what was the 'ret' value and why
'intel_iommu_assign_device' function failed.

> 
> 
> So can anyone tell me what I am doing wrong??

Please also attach the full serial long, including the Xen one - and run
it with 'loglvl=all iommu=verbose'.

Re: IOMMU and AMD 890fx

[Wei Huang]

[-- Attachment #1: Type: text/plain, Size: 4347 bytes --]

Hi Jens,

Thanks for testing it out. I tested Xen 4.0 on my machine this morning. 
The passthru did work. Here are the details:

1. I used Xen 4.0 (xen-4.0-testing.hg tree) + 2.6.18.8 Dom0
2. My system is a workstation; so it doesn't have a HDA device. Here is 
what I get from lspci (I skipped some of un-related devices):
==========
...
00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3b)
00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller
00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller
00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge
00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2 
Controller
...
01:00.0 VGA compatible controller: ATI Technologies Inc Device 949e
01:00.1 Audio device: ATI Technologies Inc RV730XT Audio device [Radeon 
HD 4670]
02:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet 
Controller (rev 06)
02:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet 
Controller (rev 06)
03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet 
Controller (Copper) (rev 06)
==========
As you can see, I have two NICs (a two-port Intel 82571EB NIC and a 
single-port 82572EI NIC). I am going to passthru 82571EB NIC to a Linux 
guest VM.

3. Here is the output from "xm dmesg". Because you also see a similar 
message, I think your IOMMU is working.
(XEN) AMD-Vi: IOMMU 0 Enabled.
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled

FYI, here is my grub entry:
==================
title 64bit Xen-unstable on SLES11
    root (hd0,5)
    kernel /boot/xen.gz dom0_mem=512M console=com2 com2=115200,8n1 iommu=1
    module /boot/vmlinuz-2.6.18.8 root=/dev/hdb6 resume=/dev/hdb3
    module /boot/initrd-2.6.18.8
==================

4. After login, I removed 82571EB from dom0

 > echo -n "0000:02:00.0" > /sys/bus/pci/drivers/e1000e/unbind
 >  echo -n "0000:02:00.1" > /sys/bus/pci/drivers/e1000e/unbind
 >  echo -n "0000:02:00.0" > /sys/bus/pci/drivers/pciback/new_slot
 >  echo -n "0000:02:00.1" > /sys/bus/pci/drivers/pciback/new_slot
 >  echo -n "0000:02:00.0" > /sys/bus/pci/drivers/pciback/bind
 >  echo -n "0000:02:00.1" > /sys/bus/pci/drivers/pciback/bind

5. In guest (ttylinux) configure file, I use the following setting.
...
pci=[ '02:00.0', '02:00.1' ]
...

6. After ttylinux booted, I saw two Intel NICs. The IP address was 
obtained automatically. See the attached image file.

Same as what Konrad has mentioned, I would recommend two things: i) 
instead of passthru HDA, try other independent PCIe devices first (such 
as NICs); ii) configure the passthru devices in guest configure file, 
instead of use it a hot-plug device. I can send you my guest configure 
file if you need it.

PS: I am pretty confident HDA passthru work because I tested it on an 
desktop system before. Here is the list of devices I have passthru to a 
Win7 guest VM on that system: (1) USB keyboard and mouse; (2) HDA audio; 
(3) ATI graphics.

Thanks,
-Wei



Konrad Rzeszutek Wilk wrote:
> On Thu, Jun 17, 2010 at 12:09:31AM +0200, Jens Krehbiel-Gräther wrote:
>   
>> Hi!
>>
>> I now have a Asus Crosshair IV for testing and this board (as I
>> postet earlier) supports iommu. Now I have installed Debian lenny
>> (5.0) and compiled xen 4.0.0.
>> Xen is running now and I have installed a hvm debian lenny to which
>> I would like to bind a pci device but that doesn't work.
>> Perhaps I am doing something wrong and you could help me?
>>     
>
> Is that the _only_ PCI device you have tried to assign? Did you try to
> assign other PCI devices which have the function number being zero?
>   
>> and xm dmesg this:
>>
>> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device
>> (0:14.2) failed
>>     
>
> That is not very useful. Wish it gave you the return code at least.
> If you are comfortable with adding printks in the Xen hypervisor I would 
> modify it a bit to see what was the 'ret' value and why
> 'intel_iommu_assign_device' function failed.
>
>   
>> So can anyone tell me what I am doing wrong??
>>     
>
> Please also attach the full serial long, including the Xen one - and run
> it with 'loglvl=all iommu=verbose'.
>
>   


[-- Attachment #2: ttylinux-passthru.jpg --]
[-- Type: image/jpeg, Size: 65275 bytes --]

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Hi!

Thanks for the tips!

With xen-4.0-testing.hg tree + 2.6.32-bpo.5-xen-amd64 (from debian lenny 
backports) it works. So the 4.0.0 release could be buggy?
When I compile and install xen-4.0-testing.hg tree it works as you 
described.
With 4.0.0 it did not work with any pci device, with xen-4.0-testing.hg 
tree it works with the two I tested for the moment:

00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) (rev 40)
03:00.0 Ethernet controller: Marvell Technology Group Ltd. Device 4381 
(rev 11)

(I just randomly take two devices).
They appear in the hvm virtual machine and they are working.

Great, thank you very much!

Jens


Am 17.06.2010 19:16, schrieb Wei Huang:
> Hi Jens,
>
> Thanks for testing it out. I tested Xen 4.0 on my machine this morning.
> The passthru did work. Here are the details:
>
> 1. I used Xen 4.0 (xen-4.0-testing.hg tree) + 2.6.18.8 Dom0
> 2. My system is a workstation; so it doesn't have a HDA device. Here is
> what I get from lspci (I skipped some of un-related devices):
> ==========
> ...
> 00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3b)
> 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller
> 00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller
> 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge
> 00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2
> Controller
> ...
> 01:00.0 VGA compatible controller: ATI Technologies Inc Device 949e
> 01:00.1 Audio device: ATI Technologies Inc RV730XT Audio device [Radeon
> HD 4670]
> 02:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
> Controller (rev 06)
> 02:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
> Controller (rev 06)
> 03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet
> Controller (Copper) (rev 06)
> ==========
> As you can see, I have two NICs (a two-port Intel 82571EB NIC and a
> single-port 82572EI NIC). I am going to passthru 82571EB NIC to a Linux
> guest VM.
>
> 3. Here is the output from "xm dmesg". Because you also see a similar
> message, I think your IOMMU is working.
> (XEN) AMD-Vi: IOMMU 0 Enabled.
> (XEN) I/O virtualisation enabled
> (XEN) I/O virtualisation for PV guests disabled
>
> FYI, here is my grub entry:
> ==================
> title 64bit Xen-unstable on SLES11
> root (hd0,5)
> kernel /boot/xen.gz dom0_mem=512M console=com2 com2=115200,8n1 iommu=1
> module /boot/vmlinuz-2.6.18.8 root=/dev/hdb6 resume=/dev/hdb3
> module /boot/initrd-2.6.18.8
> ==================
>
> 4. After login, I removed 82571EB from dom0
>
>  > echo -n "0000:02:00.0" > /sys/bus/pci/drivers/e1000e/unbind
>  > echo -n "0000:02:00.1" > /sys/bus/pci/drivers/e1000e/unbind
>  > echo -n "0000:02:00.0" > /sys/bus/pci/drivers/pciback/new_slot
>  > echo -n "0000:02:00.1" > /sys/bus/pci/drivers/pciback/new_slot
>  > echo -n "0000:02:00.0" > /sys/bus/pci/drivers/pciback/bind
>  > echo -n "0000:02:00.1" > /sys/bus/pci/drivers/pciback/bind
>
> 5. In guest (ttylinux) configure file, I use the following setting.
> ...
> pci=[ '02:00.0', '02:00.1' ]
> ...
>
> 6. After ttylinux booted, I saw two Intel NICs. The IP address was
> obtained automatically. See the attached image file.
>
> Same as what Konrad has mentioned, I would recommend two things: i)
> instead of passthru HDA, try other independent PCIe devices first (such
> as NICs); ii) configure the passthru devices in guest configure file,
> instead of use it a hot-plug device. I can send you my guest configure
> file if you need it.
>
> PS: I am pretty confident HDA passthru work because I tested it on an
> desktop system before. Here is the list of devices I have passthru to a
> Win7 guest VM on that system: (1) USB keyboard and mouse; (2) HDA audio;
> (3) ATI graphics.
>
> Thanks,
> -Wei
>
>
>
> Konrad Rzeszutek Wilk wrote:
>> On Thu, Jun 17, 2010 at 12:09:31AM +0200, Jens Krehbiel-Gräther wrote:
>>> Hi!
>>>
>>> I now have a Asus Crosshair IV for testing and this board (as I
>>> postet earlier) supports iommu. Now I have installed Debian lenny
>>> (5.0) and compiled xen 4.0.0.
>>> Xen is running now and I have installed a hvm debian lenny to which
>>> I would like to bind a pci device but that doesn't work.
>>> Perhaps I am doing something wrong and you could help me?
>>
>> Is that the _only_ PCI device you have tried to assign? Did you try to
>> assign other PCI devices which have the function number being zero?
>>> and xm dmesg this:
>>>
>>> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device
>>> (0:14.2) failed
>>
>> That is not very useful. Wish it gave you the return code at least.
>> If you are comfortable with adding printks in the Xen hypervisor I
>> would modify it a bit to see what was the 'ret' value and why
>> 'intel_iommu_assign_device' function failed.
>>
>>> So can anyone tell me what I am doing wrong??
>>
>> Please also attach the full serial long, including the Xen one - and run
>> it with 'loglvl=all iommu=verbose'.
>>
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: IOMMU and AMD 890fx

[Sander Eikelenboom]

Hi Jens,

Thx for reporting your successful pci passthrough with an AMD IOMMU, i have put this motherboard in the xen vt-d/iommu wiki as a motherboard with the BIOS supporting the IOMMU.
http://wiki.xensource.com/xenwiki/VTdHowTo

--
Sander

> Hi!

> Thanks for the tips!

> With xen-4.0-testing.hg tree + 2.6.32-bpo.5-xen-amd64 (from debian lenny 
> backports) it works. So the 4.0.0 release could be buggy?
> When I compile and install xen-4.0-testing.hg tree it works as you 
> described.
> With 4.0.0 it did not work with any pci device, with xen-4.0-testing.hg 
> tree it works with the two I tested for the moment:

> 00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) (rev 40)
> 03:00.0 Ethernet controller: Marvell Technology Group Ltd. Device 4381 
> (rev 11)

> (I just randomly take two devices).
> They appear in the hvm virtual machine and they are working.

> Great, thank you very much!

> Jens


> Am 17.06.2010 19:16, schrieb Wei Huang:
>> Hi Jens,
>>
>> Thanks for testing it out. I tested Xen 4.0 on my machine this morning.
>> The passthru did work. Here are the details:
>>
>> 1. I used Xen 4.0 (xen-4.0-testing.hg tree) + 2.6.18.8 Dom0
>> 2. My system is a workstation; so it doesn't have a HDA device. Here is
>> what I get from lspci (I skipped some of un-related devices):
>> ==========
>> ...
>> 00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3b)
>> 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller
>> 00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller
>> 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge
>> 00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2
>> Controller
>> ...
>> 01:00.0 VGA compatible controller: ATI Technologies Inc Device 949e
>> 01:00.1 Audio device: ATI Technologies Inc RV730XT Audio device [Radeon
>> HD 4670]
>> 02:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
>> Controller (rev 06)
>> 02:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
>> Controller (rev 06)
>> 03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet
>> Controller (Copper) (rev 06)
>> ==========
>> As you can see, I have two NICs (a two-port Intel 82571EB NIC and a
>> single-port 82572EI NIC). I am going to passthru 82571EB NIC to a Linux
>> guest VM.
>>
>> 3. Here is the output from "xm dmesg". Because you also see a similar
>> message, I think your IOMMU is working.
>> (XEN) AMD-Vi: IOMMU 0 Enabled.
>> (XEN) I/O virtualisation enabled
>> (XEN) I/O virtualisation for PV guests disabled
>>
>> FYI, here is my grub entry:
>> ==================
>> title 64bit Xen-unstable on SLES11
>> root (hd0,5)
>> kernel /boot/xen.gz dom0_mem=512M console=com2 com2=115200,8n1 iommu=1
>> module /boot/vmlinuz-2.6.18.8 root=/dev/hdb6 resume=/dev/hdb3
>> module /boot/initrd-2.6.18.8
>> ==================
>>
>> 4. After login, I removed 82571EB from dom0
>>
>>  > echo -n "0000:02:00.0" > /sys/bus/pci/drivers/e1000e/unbind
>>  > echo -n "0000:02:00.1" > /sys/bus/pci/drivers/e1000e/unbind
>>  > echo -n "0000:02:00.0" > /sys/bus/pci/drivers/pciback/new_slot
>>  > echo -n "0000:02:00.1" > /sys/bus/pci/drivers/pciback/new_slot
>>  > echo -n "0000:02:00.0" > /sys/bus/pci/drivers/pciback/bind
>>  > echo -n "0000:02:00.1" > /sys/bus/pci/drivers/pciback/bind
>>
>> 5. In guest (ttylinux) configure file, I use the following setting.
>> ...
>> pci=[ '02:00.0', '02:00.1' ]
>> ...
>>
>> 6. After ttylinux booted, I saw two Intel NICs. The IP address was
>> obtained automatically. See the attached image file.
>>
>> Same as what Konrad has mentioned, I would recommend two things: i)
>> instead of passthru HDA, try other independent PCIe devices first (such
>> as NICs); ii) configure the passthru devices in guest configure file,
>> instead of use it a hot-plug device. I can send you my guest configure
>> file if you need it.
>>
>> PS: I am pretty confident HDA passthru work because I tested it on an
>> desktop system before. Here is the list of devices I have passthru to a
>> Win7 guest VM on that system: (1) USB keyboard and mouse; (2) HDA audio;
>> (3) ATI graphics.
>>
>> Thanks,
>> -Wei
>>
>>
>>
>> Konrad Rzeszutek Wilk wrote:
>>> On Thu, Jun 17, 2010 at 12:09:31AM +0200, Jens Krehbiel-Gräther wrote:
>>>> Hi!
>>>>
>>>> I now have a Asus Crosshair IV for testing and this board (as I
>>>> postet earlier) supports iommu. Now I have installed Debian lenny
>>>> (5.0) and compiled xen 4.0.0.
>>>> Xen is running now and I have installed a hvm debian lenny to which
>>>> I would like to bind a pci device but that doesn't work.
>>>> Perhaps I am doing something wrong and you could help me?
>>>
>>> Is that the _only_ PCI device you have tried to assign? Did you try to
>>> assign other PCI devices which have the function number being zero?
>>>> and xm dmesg this:
>>>>
>>>> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device
>>>> (0:14.2) failed
>>>
>>> That is not very useful. Wish it gave you the return code at least.
>>> If you are comfortable with adding printks in the Xen hypervisor I
>>> would modify it a bit to see what was the 'ret' value and why
>>> 'intel_iommu_assign_device' function failed.
>>>
>>>> So can anyone tell me what I am doing wrong??
>>>
>>> Please also attach the full serial long, including the Xen one - and run
>>> it with 'loglvl=all iommu=verbose'.
>>>
>>
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel



-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it

RE: IOMMU and AMD 890fx

[Huang2, Wei]

[-- Attachment #1: Type: text/plain, Size: 5891 bytes --]

Great. Where did you get xen-4.0 from? www.xen.org/products/xen_source.html? We can certainly take a look. It might not have all IOMMU patches as in xen-4.0-testing. Wei Wang is out for vacation. So it might take a while...

Thanks,
-Wei


-----Original Message-----
From: Jens Krehbiel-Gräther [mailto:xen@jkg-it-services.de] 
Sent: Thursday, June 17, 2010 6:36 PM
To: Huang2, Wei
Cc: Konrad Rzeszutek Wilk; Przywara, Andre; xen-devel@lists.xensource.com; Leonardo Prosperi; Joerg Roedel; Eikelenboom; Joop Boonen; Oakley
Subject: Re: [Xen-devel] IOMMU and AMD 890fx

Hi!

Thanks for the tips!

With xen-4.0-testing.hg tree + 2.6.32-bpo.5-xen-amd64 (from debian lenny 
backports) it works. So the 4.0.0 release could be buggy?
When I compile and install xen-4.0-testing.hg tree it works as you 
described.
With 4.0.0 it did not work with any pci device, with xen-4.0-testing.hg 
tree it works with the two I tested for the moment:

00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) (rev 40)
03:00.0 Ethernet controller: Marvell Technology Group Ltd. Device 4381 
(rev 11)

(I just randomly take two devices).
They appear in the hvm virtual machine and they are working.

Great, thank you very much!

Jens


Am 17.06.2010 19:16, schrieb Wei Huang:
> Hi Jens,
>
> Thanks for testing it out. I tested Xen 4.0 on my machine this morning.
> The passthru did work. Here are the details:
>
> 1. I used Xen 4.0 (xen-4.0-testing.hg tree) + 2.6.18.8 Dom0
> 2. My system is a workstation; so it doesn't have a HDA device. Here is
> what I get from lspci (I skipped some of un-related devices):
> ==========
> ...
> 00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3b)
> 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller
> 00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller
> 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge
> 00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2
> Controller
> ...
> 01:00.0 VGA compatible controller: ATI Technologies Inc Device 949e
> 01:00.1 Audio device: ATI Technologies Inc RV730XT Audio device [Radeon
> HD 4670]
> 02:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
> Controller (rev 06)
> 02:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
> Controller (rev 06)
> 03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet
> Controller (Copper) (rev 06)
> ==========
> As you can see, I have two NICs (a two-port Intel 82571EB NIC and a
> single-port 82572EI NIC). I am going to passthru 82571EB NIC to a Linux
> guest VM.
>
> 3. Here is the output from "xm dmesg". Because you also see a similar
> message, I think your IOMMU is working.
> (XEN) AMD-Vi: IOMMU 0 Enabled.
> (XEN) I/O virtualisation enabled
> (XEN) I/O virtualisation for PV guests disabled
>
> FYI, here is my grub entry:
> ==================
> title 64bit Xen-unstable on SLES11
> root (hd0,5)
> kernel /boot/xen.gz dom0_mem=512M console=com2 com2=115200,8n1 iommu=1
> module /boot/vmlinuz-2.6.18.8 root=/dev/hdb6 resume=/dev/hdb3
> module /boot/initrd-2.6.18.8
> ==================
>
> 4. After login, I removed 82571EB from dom0
>
>  > echo -n "0000:02:00.0" > /sys/bus/pci/drivers/e1000e/unbind
>  > echo -n "0000:02:00.1" > /sys/bus/pci/drivers/e1000e/unbind
>  > echo -n "0000:02:00.0" > /sys/bus/pci/drivers/pciback/new_slot
>  > echo -n "0000:02:00.1" > /sys/bus/pci/drivers/pciback/new_slot
>  > echo -n "0000:02:00.0" > /sys/bus/pci/drivers/pciback/bind
>  > echo -n "0000:02:00.1" > /sys/bus/pci/drivers/pciback/bind
>
> 5. In guest (ttylinux) configure file, I use the following setting.
> ...
> pci=[ '02:00.0', '02:00.1' ]
> ...
>
> 6. After ttylinux booted, I saw two Intel NICs. The IP address was
> obtained automatically. See the attached image file.
>
> Same as what Konrad has mentioned, I would recommend two things: i)
> instead of passthru HDA, try other independent PCIe devices first (such
> as NICs); ii) configure the passthru devices in guest configure file,
> instead of use it a hot-plug device. I can send you my guest configure
> file if you need it.
>
> PS: I am pretty confident HDA passthru work because I tested it on an
> desktop system before. Here is the list of devices I have passthru to a
> Win7 guest VM on that system: (1) USB keyboard and mouse; (2) HDA audio;
> (3) ATI graphics.
>
> Thanks,
> -Wei
>
>
>
> Konrad Rzeszutek Wilk wrote:
>> On Thu, Jun 17, 2010 at 12:09:31AM +0200, Jens Krehbiel-Gräther wrote:
>>> Hi!
>>>
>>> I now have a Asus Crosshair IV for testing and this board (as I
>>> postet earlier) supports iommu. Now I have installed Debian lenny
>>> (5.0) and compiled xen 4.0.0.
>>> Xen is running now and I have installed a hvm debian lenny to which
>>> I would like to bind a pci device but that doesn't work.
>>> Perhaps I am doing something wrong and you could help me?
>>
>> Is that the _only_ PCI device you have tried to assign? Did you try to
>> assign other PCI devices which have the function number being zero?
>>> and xm dmesg this:
>>>
>>> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device
>>> (0:14.2) failed
>>
>> That is not very useful. Wish it gave you the return code at least.
>> If you are comfortable with adding printks in the Xen hypervisor I
>> would modify it a bit to see what was the 'ret' value and why
>> 'intel_iommu_assign_device' function failed.
>>
>>> So can anyone tell me what I am doing wrong??
>>
>> Please also attach the full serial long, including the Xen one - and run
>> it with 'loglvl=all iommu=verbose'.
>>
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


[-- Attachment #2: mc_patch_010000c2.bin --]
[-- Type: application/octet-stream, Size: 960 bytes --]

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Hi!

Yes xen-4.0.0 I got from http://www.xen.org/products/xen_source.html 
(donwload link is 
http://bits.xensource.com/oss-xen/release/4.0.0/xen-4.0.0.tar.gz).

Jens


Am 18.06.2010 19:36, schrieb Huang2, Wei:
> Great. Where did you get xen-4.0 from? www.xen.org/products/xen_source.html? We can certainly take a look. It might not have all IOMMU patches as in xen-4.0-testing. Wei Wang is out for vacation. So it might take a while...
>
> Thanks,
> -Wei
>
>
> -----Original Message-----
> From: Jens Krehbiel-Gräther [mailto:xen@jkg-it-services.de]
> Sent: Thursday, June 17, 2010 6:36 PM
> To: Huang2, Wei
> Cc: Konrad Rzeszutek Wilk; Przywara, Andre; xen-devel@lists.xensource.com; Leonardo Prosperi; Joerg Roedel; Eikelenboom; Joop Boonen; Oakley
> Subject: Re: [Xen-devel] IOMMU and AMD 890fx
>
> Hi!
>
> Thanks for the tips!
>
> With xen-4.0-testing.hg tree + 2.6.32-bpo.5-xen-amd64 (from debian lenny
> backports) it works. So the 4.0.0 release could be buggy?
> When I compile and install xen-4.0-testing.hg tree it works as you
> described.
> With 4.0.0 it did not work with any pci device, with xen-4.0-testing.hg
> tree it works with the two I tested for the moment:
>
> 00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) (rev 40)
> 03:00.0 Ethernet controller: Marvell Technology Group Ltd. Device 4381
> (rev 11)
>
> (I just randomly take two devices).
> They appear in the hvm virtual machine and they are working.
>
> Great, thank you very much!
>
> Jens
>
>
> Am 17.06.2010 19:16, schrieb Wei Huang:
>> Hi Jens,
>>
>> Thanks for testing it out. I tested Xen 4.0 on my machine this morning.
>> The passthru did work. Here are the details:
>>
>> 1. I used Xen 4.0 (xen-4.0-testing.hg tree) + 2.6.18.8 Dom0
>> 2. My system is a workstation; so it doesn't have a HDA device. Here is
>> what I get from lspci (I skipped some of un-related devices):
>> ==========
>> ...
>> 00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3b)
>> 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller
>> 00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller
>> 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge
>> 00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2
>> Controller
>> ...
>> 01:00.0 VGA compatible controller: ATI Technologies Inc Device 949e
>> 01:00.1 Audio device: ATI Technologies Inc RV730XT Audio device [Radeon
>> HD 4670]
>> 02:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
>> Controller (rev 06)
>> 02:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
>> Controller (rev 06)
>> 03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet
>> Controller (Copper) (rev 06)
>> ==========
>> As you can see, I have two NICs (a two-port Intel 82571EB NIC and a
>> single-port 82572EI NIC). I am going to passthru 82571EB NIC to a Linux
>> guest VM.
>>
>> 3. Here is the output from "xm dmesg". Because you also see a similar
>> message, I think your IOMMU is working.
>> (XEN) AMD-Vi: IOMMU 0 Enabled.
>> (XEN) I/O virtualisation enabled
>> (XEN) I/O virtualisation for PV guests disabled
>>
>> FYI, here is my grub entry:
>> ==================
>> title 64bit Xen-unstable on SLES11
>> root (hd0,5)
>> kernel /boot/xen.gz dom0_mem=512M console=com2 com2=115200,8n1 iommu=1
>> module /boot/vmlinuz-2.6.18.8 root=/dev/hdb6 resume=/dev/hdb3
>> module /boot/initrd-2.6.18.8
>> ==================
>>
>> 4. After login, I removed 82571EB from dom0
>>
>>   >  echo -n "0000:02:00.0">  /sys/bus/pci/drivers/e1000e/unbind
>>   >  echo -n "0000:02:00.1">  /sys/bus/pci/drivers/e1000e/unbind
>>   >  echo -n "0000:02:00.0">  /sys/bus/pci/drivers/pciback/new_slot
>>   >  echo -n "0000:02:00.1">  /sys/bus/pci/drivers/pciback/new_slot
>>   >  echo -n "0000:02:00.0">  /sys/bus/pci/drivers/pciback/bind
>>   >  echo -n "0000:02:00.1">  /sys/bus/pci/drivers/pciback/bind
>>
>> 5. In guest (ttylinux) configure file, I use the following setting.
>> ...
>> pci=[ '02:00.0', '02:00.1' ]
>> ...
>>
>> 6. After ttylinux booted, I saw two Intel NICs. The IP address was
>> obtained automatically. See the attached image file.
>>
>> Same as what Konrad has mentioned, I would recommend two things: i)
>> instead of passthru HDA, try other independent PCIe devices first (such
>> as NICs); ii) configure the passthru devices in guest configure file,
>> instead of use it a hot-plug device. I can send you my guest configure
>> file if you need it.
>>
>> PS: I am pretty confident HDA passthru work because I tested it on an
>> desktop system before. Here is the list of devices I have passthru to a
>> Win7 guest VM on that system: (1) USB keyboard and mouse; (2) HDA audio;
>> (3) ATI graphics.
>>
>> Thanks,
>> -Wei
>>
>>
>>
>> Konrad Rzeszutek Wilk wrote:
>>> On Thu, Jun 17, 2010 at 12:09:31AM +0200, Jens Krehbiel-Gräther wrote:
>>>> Hi!
>>>>
>>>> I now have a Asus Crosshair IV for testing and this board (as I
>>>> postet earlier) supports iommu. Now I have installed Debian lenny
>>>> (5.0) and compiled xen 4.0.0.
>>>> Xen is running now and I have installed a hvm debian lenny to which
>>>> I would like to bind a pci device but that doesn't work.
>>>> Perhaps I am doing something wrong and you could help me?
>>>
>>> Is that the _only_ PCI device you have tried to assign? Did you try to
>>> assign other PCI devices which have the function number being zero?
>>>> and xm dmesg this:
>>>>
>>>> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device
>>>> (0:14.2) failed
>>>
>>> That is not very useful. Wish it gave you the return code at least.
>>> If you are comfortable with adding printks in the Xen hypervisor I
>>> would modify it a bit to see what was the 'ret' value and why
>>> 'intel_iommu_assign_device' function failed.
>>>
>>>> So can anyone tell me what I am doing wrong??
>>>
>>> Please also attach the full serial long, including the Xen one - and run
>>> it with 'loglvl=all iommu=verbose'.
>>>
>>
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Hi!

OK on monday I can tell you more about the asus board M4A89TD (I will 
have one for testing).

Jens


Am 18.06.2010 16:24, schrieb Sander Eikelenboom:
> Hi Jens,
>
> Thx for reporting your successful pci passthrough with an AMD IOMMU, i have put this motherboard in the xen vt-d/iommu wiki as a motherboard with the BIOS supporting the IOMMU.
> http://wiki.xensource.com/xenwiki/VTdHowTo
>
> --
> Sander
>
>> Hi!
>
>> Thanks for the tips!
>
>> With xen-4.0-testing.hg tree + 2.6.32-bpo.5-xen-amd64 (from debian lenny
>> backports) it works. So the 4.0.0 release could be buggy?
>> When I compile and install xen-4.0-testing.hg tree it works as you
>> described.
>> With 4.0.0 it did not work with any pci device, with xen-4.0-testing.hg
>> tree it works with the two I tested for the moment:
>
>> 00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) (rev 40)
>> 03:00.0 Ethernet controller: Marvell Technology Group Ltd. Device 4381
>> (rev 11)
>
>> (I just randomly take two devices).
>> They appear in the hvm virtual machine and they are working.
>
>> Great, thank you very much!
>
>> Jens
>
>
>> Am 17.06.2010 19:16, schrieb Wei Huang:
>>> Hi Jens,
>>>
>>> Thanks for testing it out. I tested Xen 4.0 on my machine this morning.
>>> The passthru did work. Here are the details:
>>>
>>> 1. I used Xen 4.0 (xen-4.0-testing.hg tree) + 2.6.18.8 Dom0
>>> 2. My system is a workstation; so it doesn't have a HDA device. Here is
>>> what I get from lspci (I skipped some of un-related devices):
>>> ==========
>>> ...
>>> 00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3b)
>>> 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller
>>> 00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller
>>> 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge
>>> 00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2
>>> Controller
>>> ...
>>> 01:00.0 VGA compatible controller: ATI Technologies Inc Device 949e
>>> 01:00.1 Audio device: ATI Technologies Inc RV730XT Audio device [Radeon
>>> HD 4670]
>>> 02:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
>>> Controller (rev 06)
>>> 02:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
>>> Controller (rev 06)
>>> 03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet
>>> Controller (Copper) (rev 06)
>>> ==========
>>> As you can see, I have two NICs (a two-port Intel 82571EB NIC and a
>>> single-port 82572EI NIC). I am going to passthru 82571EB NIC to a Linux
>>> guest VM.
>>>
>>> 3. Here is the output from "xm dmesg". Because you also see a similar
>>> message, I think your IOMMU is working.
>>> (XEN) AMD-Vi: IOMMU 0 Enabled.
>>> (XEN) I/O virtualisation enabled
>>> (XEN) I/O virtualisation for PV guests disabled
>>>
>>> FYI, here is my grub entry:
>>> ==================
>>> title 64bit Xen-unstable on SLES11
>>> root (hd0,5)
>>> kernel /boot/xen.gz dom0_mem=512M console=com2 com2=115200,8n1 iommu=1
>>> module /boot/vmlinuz-2.6.18.8 root=/dev/hdb6 resume=/dev/hdb3
>>> module /boot/initrd-2.6.18.8
>>> ==================
>>>
>>> 4. After login, I removed 82571EB from dom0
>>>
>>>   >  echo -n "0000:02:00.0">  /sys/bus/pci/drivers/e1000e/unbind
>>>   >  echo -n "0000:02:00.1">  /sys/bus/pci/drivers/e1000e/unbind
>>>   >  echo -n "0000:02:00.0">  /sys/bus/pci/drivers/pciback/new_slot
>>>   >  echo -n "0000:02:00.1">  /sys/bus/pci/drivers/pciback/new_slot
>>>   >  echo -n "0000:02:00.0">  /sys/bus/pci/drivers/pciback/bind
>>>   >  echo -n "0000:02:00.1">  /sys/bus/pci/drivers/pciback/bind
>>>
>>> 5. In guest (ttylinux) configure file, I use the following setting.
>>> ...
>>> pci=[ '02:00.0', '02:00.1' ]
>>> ...
>>>
>>> 6. After ttylinux booted, I saw two Intel NICs. The IP address was
>>> obtained automatically. See the attached image file.
>>>
>>> Same as what Konrad has mentioned, I would recommend two things: i)
>>> instead of passthru HDA, try other independent PCIe devices first (such
>>> as NICs); ii) configure the passthru devices in guest configure file,
>>> instead of use it a hot-plug device. I can send you my guest configure
>>> file if you need it.
>>>
>>> PS: I am pretty confident HDA passthru work because I tested it on an
>>> desktop system before. Here is the list of devices I have passthru to a
>>> Win7 guest VM on that system: (1) USB keyboard and mouse; (2) HDA audio;
>>> (3) ATI graphics.
>>>
>>> Thanks,
>>> -Wei
>>>
>>>
>>>
>>> Konrad Rzeszutek Wilk wrote:
>>>> On Thu, Jun 17, 2010 at 12:09:31AM +0200, Jens Krehbiel-Gräther wrote:
>>>>> Hi!
>>>>>
>>>>> I now have a Asus Crosshair IV for testing and this board (as I
>>>>> postet earlier) supports iommu. Now I have installed Debian lenny
>>>>> (5.0) and compiled xen 4.0.0.
>>>>> Xen is running now and I have installed a hvm debian lenny to which
>>>>> I would like to bind a pci device but that doesn't work.
>>>>> Perhaps I am doing something wrong and you could help me?
>>>>
>>>> Is that the _only_ PCI device you have tried to assign? Did you try to
>>>> assign other PCI devices which have the function number being zero?
>>>>> and xm dmesg this:
>>>>>
>>>>> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device
>>>>> (0:14.2) failed
>>>>
>>>> That is not very useful. Wish it gave you the return code at least.
>>>> If you are comfortable with adding printks in the Xen hypervisor I
>>>> would modify it a bit to see what was the 'ret' value and why
>>>> 'intel_iommu_assign_device' function failed.
>>>>
>>>>> So can anyone tell me what I am doing wrong??
>>>>
>>>> Please also attach the full serial long, including the Xen one - and run
>>>> it with 'loglvl=all iommu=verbose'.
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>
>
>

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Another board with working AMD IOMMU: Asus M4A89TD Pro/USB3

Just got it in hands and testet successful!

Jens



Am 18.06.2010 16:24, schrieb Sander Eikelenboom:
> Hi Jens,
>
> Thx for reporting your successful pci passthrough with an AMD IOMMU, i have put this motherboard in the xen vt-d/iommu wiki as a motherboard with the BIOS supporting the IOMMU.
> http://wiki.xensource.com/xenwiki/VTdHowTo
>
> --
> Sander
>
>> Hi!
>
>> Thanks for the tips!
>
>> With xen-4.0-testing.hg tree + 2.6.32-bpo.5-xen-amd64 (from debian lenny
>> backports) it works. So the 4.0.0 release could be buggy?
>> When I compile and install xen-4.0-testing.hg tree it works as you
>> described.
>> With 4.0.0 it did not work with any pci device, with xen-4.0-testing.hg
>> tree it works with the two I tested for the moment:
>
>> 00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) (rev 40)
>> 03:00.0 Ethernet controller: Marvell Technology Group Ltd. Device 4381
>> (rev 11)
>
>> (I just randomly take two devices).
>> They appear in the hvm virtual machine and they are working.
>
>> Great, thank you very much!
>
>> Jens
>
>
>> Am 17.06.2010 19:16, schrieb Wei Huang:
>>> Hi Jens,
>>>
>>> Thanks for testing it out. I tested Xen 4.0 on my machine this morning.
>>> The passthru did work. Here are the details:
>>>
>>> 1. I used Xen 4.0 (xen-4.0-testing.hg tree) + 2.6.18.8 Dom0
>>> 2. My system is a workstation; so it doesn't have a HDA device. Here is
>>> what I get from lspci (I skipped some of un-related devices):
>>> ==========
>>> ...
>>> 00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3b)
>>> 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller
>>> 00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller
>>> 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge
>>> 00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2
>>> Controller
>>> ...
>>> 01:00.0 VGA compatible controller: ATI Technologies Inc Device 949e
>>> 01:00.1 Audio device: ATI Technologies Inc RV730XT Audio device [Radeon
>>> HD 4670]
>>> 02:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
>>> Controller (rev 06)
>>> 02:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
>>> Controller (rev 06)
>>> 03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet
>>> Controller (Copper) (rev 06)
>>> ==========
>>> As you can see, I have two NICs (a two-port Intel 82571EB NIC and a
>>> single-port 82572EI NIC). I am going to passthru 82571EB NIC to a Linux
>>> guest VM.
>>>
>>> 3. Here is the output from "xm dmesg". Because you also see a similar
>>> message, I think your IOMMU is working.
>>> (XEN) AMD-Vi: IOMMU 0 Enabled.
>>> (XEN) I/O virtualisation enabled
>>> (XEN) I/O virtualisation for PV guests disabled
>>>
>>> FYI, here is my grub entry:
>>> ==================
>>> title 64bit Xen-unstable on SLES11
>>> root (hd0,5)
>>> kernel /boot/xen.gz dom0_mem=512M console=com2 com2=115200,8n1 iommu=1
>>> module /boot/vmlinuz-2.6.18.8 root=/dev/hdb6 resume=/dev/hdb3
>>> module /boot/initrd-2.6.18.8
>>> ==================
>>>
>>> 4. After login, I removed 82571EB from dom0
>>>
>>>   >  echo -n "0000:02:00.0">  /sys/bus/pci/drivers/e1000e/unbind
>>>   >  echo -n "0000:02:00.1">  /sys/bus/pci/drivers/e1000e/unbind
>>>   >  echo -n "0000:02:00.0">  /sys/bus/pci/drivers/pciback/new_slot
>>>   >  echo -n "0000:02:00.1">  /sys/bus/pci/drivers/pciback/new_slot
>>>   >  echo -n "0000:02:00.0">  /sys/bus/pci/drivers/pciback/bind
>>>   >  echo -n "0000:02:00.1">  /sys/bus/pci/drivers/pciback/bind
>>>
>>> 5. In guest (ttylinux) configure file, I use the following setting.
>>> ...
>>> pci=[ '02:00.0', '02:00.1' ]
>>> ...
>>>
>>> 6. After ttylinux booted, I saw two Intel NICs. The IP address was
>>> obtained automatically. See the attached image file.
>>>
>>> Same as what Konrad has mentioned, I would recommend two things: i)
>>> instead of passthru HDA, try other independent PCIe devices first (such
>>> as NICs); ii) configure the passthru devices in guest configure file,
>>> instead of use it a hot-plug device. I can send you my guest configure
>>> file if you need it.
>>>
>>> PS: I am pretty confident HDA passthru work because I tested it on an
>>> desktop system before. Here is the list of devices I have passthru to a
>>> Win7 guest VM on that system: (1) USB keyboard and mouse; (2) HDA audio;
>>> (3) ATI graphics.
>>>
>>> Thanks,
>>> -Wei
>>>
>>>
>>>
>>> Konrad Rzeszutek Wilk wrote:
>>>> On Thu, Jun 17, 2010 at 12:09:31AM +0200, Jens Krehbiel-Gräther wrote:
>>>>> Hi!
>>>>>
>>>>> I now have a Asus Crosshair IV for testing and this board (as I
>>>>> postet earlier) supports iommu. Now I have installed Debian lenny
>>>>> (5.0) and compiled xen 4.0.0.
>>>>> Xen is running now and I have installed a hvm debian lenny to which
>>>>> I would like to bind a pci device but that doesn't work.
>>>>> Perhaps I am doing something wrong and you could help me?
>>>>
>>>> Is that the _only_ PCI device you have tried to assign? Did you try to
>>>> assign other PCI devices which have the function number being zero?
>>>>> and xm dmesg this:
>>>>>
>>>>> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device
>>>>> (0:14.2) failed
>>>>
>>>> That is not very useful. Wish it gave you the return code at least.
>>>> If you are comfortable with adding printks in the Xen hypervisor I
>>>> would modify it a bit to see what was the 'ret' value and why
>>>> 'intel_iommu_assign_device' function failed.
>>>>
>>>>> So can anyone tell me what I am doing wrong??
>>>>
>>>> Please also attach the full serial long, including the Xen one - and run
>>>> it with 'loglvl=all iommu=verbose'.
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>
>
>

Re: IOMMU and AMD 890fx

[Joerg Roedel]

Hi Jens,

great to know. Thanks for your testing.

Joerg

On Mon, Jun 21, 2010 at 05:00:10PM +0200, Jens Krehbiel-Gräther wrote:
> Another board with working AMD IOMMU: Asus M4A89TD Pro/USB3
>
> Just got it in hands and testet successful!
>
> Jens
>
>
>
> Am 18.06.2010 16:24, schrieb Sander Eikelenboom:
>> Hi Jens,
>>
>> Thx for reporting your successful pci passthrough with an AMD IOMMU, i have put this motherboard in the xen vt-d/iommu wiki as a motherboard with the BIOS supporting the IOMMU.
>> http://wiki.xensource.com/xenwiki/VTdHowTo
>>
>> --
>> Sander
>>
>>> Hi!
>>
>>> Thanks for the tips!
>>
>>> With xen-4.0-testing.hg tree + 2.6.32-bpo.5-xen-amd64 (from debian lenny
>>> backports) it works. So the 4.0.0 release could be buggy?
>>> When I compile and install xen-4.0-testing.hg tree it works as you
>>> described.
>>> With 4.0.0 it did not work with any pci device, with xen-4.0-testing.hg
>>> tree it works with the two I tested for the moment:
>>
>>> 00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) (rev 40)
>>> 03:00.0 Ethernet controller: Marvell Technology Group Ltd. Device 4381
>>> (rev 11)
>>
>>> (I just randomly take two devices).
>>> They appear in the hvm virtual machine and they are working.
>>
>>> Great, thank you very much!
>>
>>> Jens
>>
>>
>>> Am 17.06.2010 19:16, schrieb Wei Huang:
>>>> Hi Jens,
>>>>
>>>> Thanks for testing it out. I tested Xen 4.0 on my machine this morning.
>>>> The passthru did work. Here are the details:
>>>>
>>>> 1. I used Xen 4.0 (xen-4.0-testing.hg tree) + 2.6.18.8 Dom0
>>>> 2. My system is a workstation; so it doesn't have a HDA device. Here is
>>>> what I get from lspci (I skipped some of un-related devices):
>>>> ==========
>>>> ...
>>>> 00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3b)
>>>> 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller
>>>> 00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller
>>>> 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge
>>>> 00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2
>>>> Controller
>>>> ...
>>>> 01:00.0 VGA compatible controller: ATI Technologies Inc Device 949e
>>>> 01:00.1 Audio device: ATI Technologies Inc RV730XT Audio device [Radeon
>>>> HD 4670]
>>>> 02:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
>>>> Controller (rev 06)
>>>> 02:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
>>>> Controller (rev 06)
>>>> 03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet
>>>> Controller (Copper) (rev 06)
>>>> ==========
>>>> As you can see, I have two NICs (a two-port Intel 82571EB NIC and a
>>>> single-port 82572EI NIC). I am going to passthru 82571EB NIC to a Linux
>>>> guest VM.
>>>>
>>>> 3. Here is the output from "xm dmesg". Because you also see a similar
>>>> message, I think your IOMMU is working.
>>>> (XEN) AMD-Vi: IOMMU 0 Enabled.
>>>> (XEN) I/O virtualisation enabled
>>>> (XEN) I/O virtualisation for PV guests disabled
>>>>
>>>> FYI, here is my grub entry:
>>>> ==================
>>>> title 64bit Xen-unstable on SLES11
>>>> root (hd0,5)
>>>> kernel /boot/xen.gz dom0_mem=512M console=com2 com2=115200,8n1 iommu=1
>>>> module /boot/vmlinuz-2.6.18.8 root=/dev/hdb6 resume=/dev/hdb3
>>>> module /boot/initrd-2.6.18.8
>>>> ==================
>>>>
>>>> 4. After login, I removed 82571EB from dom0
>>>>
>>>>   >  echo -n "0000:02:00.0">  /sys/bus/pci/drivers/e1000e/unbind
>>>>   >  echo -n "0000:02:00.1">  /sys/bus/pci/drivers/e1000e/unbind
>>>>   >  echo -n "0000:02:00.0">  /sys/bus/pci/drivers/pciback/new_slot
>>>>   >  echo -n "0000:02:00.1">  /sys/bus/pci/drivers/pciback/new_slot
>>>>   >  echo -n "0000:02:00.0">  /sys/bus/pci/drivers/pciback/bind
>>>>   >  echo -n "0000:02:00.1">  /sys/bus/pci/drivers/pciback/bind
>>>>
>>>> 5. In guest (ttylinux) configure file, I use the following setting.
>>>> ...
>>>> pci=[ '02:00.0', '02:00.1' ]
>>>> ...
>>>>
>>>> 6. After ttylinux booted, I saw two Intel NICs. The IP address was
>>>> obtained automatically. See the attached image file.
>>>>
>>>> Same as what Konrad has mentioned, I would recommend two things: i)
>>>> instead of passthru HDA, try other independent PCIe devices first (such
>>>> as NICs); ii) configure the passthru devices in guest configure file,
>>>> instead of use it a hot-plug device. I can send you my guest configure
>>>> file if you need it.
>>>>
>>>> PS: I am pretty confident HDA passthru work because I tested it on an
>>>> desktop system before. Here is the list of devices I have passthru to a
>>>> Win7 guest VM on that system: (1) USB keyboard and mouse; (2) HDA audio;
>>>> (3) ATI graphics.
>>>>
>>>> Thanks,
>>>> -Wei
>>>>
>>>>
>>>>
>>>> Konrad Rzeszutek Wilk wrote:
>>>>> On Thu, Jun 17, 2010 at 12:09:31AM +0200, Jens Krehbiel-Gräther wrote:
>>>>>> Hi!
>>>>>>
>>>>>> I now have a Asus Crosshair IV for testing and this board (as I
>>>>>> postet earlier) supports iommu. Now I have installed Debian lenny
>>>>>> (5.0) and compiled xen 4.0.0.
>>>>>> Xen is running now and I have installed a hvm debian lenny to which
>>>>>> I would like to bind a pci device but that doesn't work.
>>>>>> Perhaps I am doing something wrong and you could help me?
>>>>>
>>>>> Is that the _only_ PCI device you have tried to assign? Did you try to
>>>>> assign other PCI devices which have the function number being zero?
>>>>>> and xm dmesg this:
>>>>>>
>>>>>> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device
>>>>>> (0:14.2) failed
>>>>>
>>>>> That is not very useful. Wish it gave you the return code at least.
>>>>> If you are comfortable with adding printks in the Xen hypervisor I
>>>>> would modify it a bit to see what was the 'ret' value and why
>>>>> 'intel_iommu_assign_device' function failed.
>>>>>
>>>>>> So can anyone tell me what I am doing wrong??
>>>>>
>>>>> Please also attach the full serial long, including the Xen one - and run
>>>>> it with 'loglvl=all iommu=verbose'.
>>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Xen-devel mailing list
>>>> Xen-devel@lists.xensource.com
>>>> http://lists.xensource.com/xen-devel
>>
>>
>>

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Asrock will support iommu in next official bios release!

Jens


Am 22.06.2010 18:36, schrieb Huang2, Wei:
> Wow, this is certainly very nice. Thanks for chasing this issue and sharing the BIOS...
>
> -Wei
>
> -----Original Message-----
> From: Jens Krehbiel-Gräther [mailto:xen@jkg-it-services.de]
> Sent: Tuesday, June 22, 2010 11:30 AM
> To: Joerg Roedel
> Cc: Leonardo Prosperi; xen-devel@lists.xensource.com; Konrad Rzeszutek Wilk; Przywara, Andre; Huang2, Wei; Sander Eikelenboom; Joop Boonen; Oakley
> Subject: Re: [Xen-devel] IOMMU and AMD 890fx
>
> Hi!
>
> Back again with information from asrock support. They send me a custom bios for the "Asrock 890FX Deluxe3" today with iommu option. I flashed it and testet and it works.
>
> Asking asrock support for distribution of this bios they said:
>
> "
> Yes, you can spread it :-)
> But please keep in mind it is right now a beta/test BIOS version, so we cannot give guarantee everything working stable... you know ;-) "
>
> So I put it as attachement to this mail. Maybe you want to add this board (with this bios as download) to the wiki list of working mainboards?
>
> Regards,
>     Jens
>
>
> Am 21.06.2010 17:39, schrieb Joerg Roedel:
>> Hi Jens,
>>
>> great to know. Thanks for your testing.
>>
>> Joerg
>>
>> On Mon, Jun 21, 2010 at 05:00:10PM +0200, Jens Krehbiel-Gräther wrote:
>>> Another board with working AMD IOMMU: Asus M4A89TD Pro/USB3
>>>
>>> Just got it in hands and testet successful!
>>>
>>> Jens
>>>
>>>
>>>
>>> Am 18.06.2010 16:24, schrieb Sander Eikelenboom:
>>>> Hi Jens,
>>>>
>>>> Thx for reporting your successful pci passthrough with an AMD IOMMU, i have put this motherboard in the xen vt-d/iommu wiki as a motherboard with the BIOS supporting the IOMMU.
>>>> http://wiki.xensource.com/xenwiki/VTdHowTo
>>>>
>>>> --
>>>> Sander
>>>>
>>>>> Hi!
>>>>
>>>>> Thanks for the tips!
>>>>
>>>>> With xen-4.0-testing.hg tree + 2.6.32-bpo.5-xen-amd64 (from debian
>>>>> lenny
>>>>> backports) it works. So the 4.0.0 release could be buggy?
>>>>> When I compile and install xen-4.0-testing.hg tree it works as you
>>>>> described.
>>>>> With 4.0.0 it did not work with any pci device, with
>>>>> xen-4.0-testing.hg tree it works with the two I tested for the moment:
>>>>
>>>>> 00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA)
>>>>> (rev 40) 03:00.0 Ethernet controller: Marvell Technology Group Ltd.
>>>>> Device 4381 (rev 11)
>>>>
>>>>> (I just randomly take two devices).
>>>>> They appear in the hvm virtual machine and they are working.
>>>>
>>>>> Great, thank you very much!
>>>>
>>>>> Jens
>>>>
>>>>
>>>>> Am 17.06.2010 19:16, schrieb Wei Huang:
>>>>>> Hi Jens,
>>>>>>
>>>>>> Thanks for testing it out. I tested Xen 4.0 on my machine this morning.
>>>>>> The passthru did work. Here are the details:
>>>>>>
>>>>>> 1. I used Xen 4.0 (xen-4.0-testing.hg tree) + 2.6.18.8 Dom0 2. My
>>>>>> system is a workstation; so it doesn't have a HDA device. Here is
>>>>>> what I get from lspci (I skipped some of un-related devices):
>>>>>> ==========
>>>>>> ...
>>>>>> 00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev
>>>>>> 3b)
>>>>>> 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE
>>>>>> Controller
>>>>>> 00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host
>>>>>> controller
>>>>>> 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge
>>>>>> 00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2
>>>>>> Controller ...
>>>>>> 01:00.0 VGA compatible controller: ATI Technologies Inc Device
>>>>>> 949e
>>>>>> 01:00.1 Audio device: ATI Technologies Inc RV730XT Audio device
>>>>>> [Radeon HD 4670] 02:00.0 Ethernet controller: Intel Corporation
>>>>>> 82571EB Gigabit Ethernet Controller (rev 06)
>>>>>> 02:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit
>>>>>> Ethernet Controller (rev 06) 03:00.0 Ethernet controller: Intel
>>>>>> Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)
>>>>>> ========== As you can see, I have two NICs (a two-port Intel
>>>>>> 82571EB NIC and a single-port 82572EI NIC). I am going to passthru
>>>>>> 82571EB NIC to a Linux guest VM.
>>>>>>
>>>>>> 3. Here is the output from "xm dmesg". Because you also see a
>>>>>> similar message, I think your IOMMU is working.
>>>>>> (XEN) AMD-Vi: IOMMU 0 Enabled.
>>>>>> (XEN) I/O virtualisation enabled
>>>>>> (XEN) I/O virtualisation for PV guests disabled
>>>>>>
>>>>>> FYI, here is my grub entry:
>>>>>> ==================
>>>>>> title 64bit Xen-unstable on SLES11 root (hd0,5) kernel
>>>>>> /boot/xen.gz dom0_mem=512M console=com2 com2=115200,8n1 iommu=1
>>>>>> module /boot/vmlinuz-2.6.18.8 root=/dev/hdb6 resume=/dev/hdb3
>>>>>> module /boot/initrd-2.6.18.8 ==================
>>>>>>
>>>>>> 4. After login, I removed 82571EB from dom0
>>>>>>
>>>>>>     >    echo -n "0000:02:00.0">    /sys/bus/pci/drivers/e1000e/unbind
>>>>>>     >    echo -n "0000:02:00.1">    /sys/bus/pci/drivers/e1000e/unbind
>>>>>>     >    echo -n "0000:02:00.0">    /sys/bus/pci/drivers/pciback/new_slot
>>>>>>     >    echo -n "0000:02:00.1">    /sys/bus/pci/drivers/pciback/new_slot
>>>>>>     >    echo -n "0000:02:00.0">    /sys/bus/pci/drivers/pciback/bind
>>>>>>     >    echo -n "0000:02:00.1">    /sys/bus/pci/drivers/pciback/bind
>>>>>>
>>>>>> 5. In guest (ttylinux) configure file, I use the following setting.
>>>>>> ...
>>>>>> pci=[ '02:00.0', '02:00.1' ]
>>>>>> ...
>>>>>>
>>>>>> 6. After ttylinux booted, I saw two Intel NICs. The IP address was
>>>>>> obtained automatically. See the attached image file.
>>>>>>
>>>>>> Same as what Konrad has mentioned, I would recommend two things:
>>>>>> i) instead of passthru HDA, try other independent PCIe devices
>>>>>> first (such as NICs); ii) configure the passthru devices in guest
>>>>>> configure file, instead of use it a hot-plug device. I can send
>>>>>> you my guest configure file if you need it.
>>>>>>
>>>>>> PS: I am pretty confident HDA passthru work because I tested it on
>>>>>> an desktop system before. Here is the list of devices I have
>>>>>> passthru to a
>>>>>> Win7 guest VM on that system: (1) USB keyboard and mouse; (2) HDA
>>>>>> audio;
>>>>>> (3) ATI graphics.
>>>>>>
>>>>>> Thanks,
>>>>>> -Wei
>>>>>>
>>>>>>
>>>>>>
>>>>>> Konrad Rzeszutek Wilk wrote:
>>>>>>> On Thu, Jun 17, 2010 at 12:09:31AM +0200, Jens Krehbiel-Gräther wrote:
>>>>>>>> Hi!
>>>>>>>>
>>>>>>>> I now have a Asus Crosshair IV for testing and this board (as I
>>>>>>>> postet earlier) supports iommu. Now I have installed Debian
>>>>>>>> lenny
>>>>>>>> (5.0) and compiled xen 4.0.0.
>>>>>>>> Xen is running now and I have installed a hvm debian lenny to
>>>>>>>> which I would like to bind a pci device but that doesn't work.
>>>>>>>> Perhaps I am doing something wrong and you could help me?
>>>>>>>
>>>>>>> Is that the _only_ PCI device you have tried to assign? Did you
>>>>>>> try to assign other PCI devices which have the function number being zero?
>>>>>>>> and xm dmesg this:
>>>>>>>>
>>>>>>>> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device
>>>>>>>> (0:14.2) failed
>>>>>>>
>>>>>>> That is not very useful. Wish it gave you the return code at least.
>>>>>>> If you are comfortable with adding printks in the Xen hypervisor
>>>>>>> I would modify it a bit to see what was the 'ret' value and why
>>>>>>> 'intel_iommu_assign_device' function failed.
>>>>>>>
>>>>>>>> So can anyone tell me what I am doing wrong??
>>>>>>>
>>>>>>> Please also attach the full serial long, including the Xen one -
>>>>>>> and run it with 'loglvl=all iommu=verbose'.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Xen-devel mailing list
>>>>>> Xen-devel@lists.xensource.com
>>>>>> http://lists.xensource.com/xen-devel
>>>>
>>>>
>>>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>

Re: IOMMU and AMD 890fx

[Joop Boonen]

I've just got the Biostar mainboard.

I can cant find any option for enabling IOMMU.

When i check via:
ls /sys/firmware/acpi/tables/
APIC  DSDT  FACP  FACS  HPET  MCFG  OEMB  SRAT  SSDT  dynamic

dmesg |grep -i iommu

lspci -d 1002:5a23

I don't see any traces of IOMMU.

I'm going to contact Biostar about this, I hope I can per sway them to
enable IOMMU.

Regards,

Joop.


On 06/15/2010 11:37 PM, Joop Boonen wrote:
> I've got the following answer from Biostar in respect to the Biostar
> TA890FXE main board.
> http://www.biostar.com.tw/app/en/mb/introduction.php?S_ID=482
>
> <quote>
> The motherboard supports IOMMU, but the CPU you are installing also must
> support IOMMU.
>
>
>
> support@biostar-usa.com
> 626-581-1055 menu select 2 
> </quote>
>
>
> I've just took the risk to order this mainboard as the main board is
> relatively cheap and I hope/presume the IOMMU will be supported
> eventually if it's not supported yet.
> I'll update you when I know the result.
> I understood IOMMU is not related to the CPU i.e. IOMMU should also be
> supported when the CPU on the main board is a AMD Phenom II X2 550 Black
> Edition?
>
> On 06/14/2010 03:49 PM, Andre Przywara wrote:
>   
>> Jens Krehbiel-Gräther wrote:
>>     
>>> Hi!
>>>
>>> I now got a Asus Crosshair IV mainboard and here are the outputs
>>> after I enabled the IOMMU option in the bios:
>>>
>>> xen-01:~# ls /sys/firmware/acpi/tables/
>>> APIC  DSDT  dynamic  FACP  FACS  HPET  IVRS  MCFG  OEMB  SRAT  SSDT
>>> xen-01:~# dmesg |grep -i iommu
>>> [    1.691432] AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
>>> xen-01:~# lspci -d 1002:5a23
>>> 00:00.2 Generic system peripheral [0806]: ATI Technologies Inc Device
>>> 5a23
>>> xen-01:~#
>>>
>>> So here is the IOMMU of the 890FX. Will go further for testing now..
>>>       
>> Great news, thanks for pursuing that.
>>
>> For the other boards:
>> We have identified an issue and are working on a solution. Stay tuned.
>>
>> Also please note that sometimes board vendors also call the AMD64 GART
>> an IOMMU, so if you have correspondence with them, make sure you are
>> talking about the same thing.
>>
>> Regards,
>> Andre.
>>
>>     
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>   

Re: IOMMU and AMD 890fx

[Andrew Oakley]

On 10/06/10 19:49, Andrew Oakley wrote:
> I have a Gigabyte GA-890FXA-UD5 motherboard.  I have opened a support
> ticket with Gigabyte but have not yet received any reply.  The
> virtualization option in the BIOS is turned on, there do not appear to
> be any other relevant options.  
> 
> $ lspci -d 1002:5a23
> $ ls /sys/firmware/acpi/tables/
> APIC  DSDT  dynamic  FACP  FACS  HPET  MATS  MCFG  SSDT  TAMG
> 
> If there is any response from Gigabyte to AMD can it be published on
> the list please?

Good news everyone, Gigabyte have now implemented this feature, see
http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/

-- 
Andrew Oakley

Re: IOMMU and AMD 890fx

[Sander Eikelenboom]

Hello Andrew,

That's good news, although the bios seems still in beta.
I have updated the wiki that it seems to work with beta bioses for both the asrock and the gigabyte mobo's with beta-bioses.

Seems MSI and Biostar are the ones left behind at the moment.

Please do report when the final bioses come out, and if you have been able to confirm the IOMMU working by your own experience !

--

Sander


Tuesday, July 6, 2010, 4:54:16 PM, you wrote:

> On 10/06/10 19:49, Andrew Oakley wrote:
>> I have a Gigabyte GA-890FXA-UD5 motherboard.  I have opened a support
>> ticket with Gigabyte but have not yet received any reply.  The
>> virtualization option in the BIOS is turned on, there do not appear to
>> be any other relevant options.  
>> 
>> $ lspci -d 1002:5a23
>> $ ls /sys/firmware/acpi/tables/
>> APIC  DSDT  dynamic  FACP  FACS  HPET  MATS  MCFG  SSDT  TAMG
>> 
>> If there is any response from Gigabyte to AMD can it be published on
>> the list please?

> Good news everyone, Gigabyte have now implemented this feature, see
> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/




-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it

Re: IOMMU and AMD 890fx

[Sander Eikelenboom]

Hi all,

I have received a beta-bios from MSI for the MSI 890FXA-GD70 that should support the IOMMU.
Since I don't have this board (yet), if there is anyone who has it and cares to test, just drop an email and I can forward the bios.

If it works, that would only leave Biostar not supporting it, and with Asrock, Gigabyte and MSI now in beta.
--

Sander

Tuesday, July 6, 2010, 4:54:16 PM, you wrote:

> On 10/06/10 19:49, Andrew Oakley wrote:
>> I have a Gigabyte GA-890FXA-UD5 motherboard.  I have opened a support
>> ticket with Gigabyte but have not yet received any reply.  The
>> virtualization option in the BIOS is turned on, there do not appear to
>> be any other relevant options.  
>> 
>> $ lspci -d 1002:5a23
>> $ ls /sys/firmware/acpi/tables/
>> APIC  DSDT  dynamic  FACP  FACS  HPET  MATS  MCFG  SSDT  TAMG
>> 
>> If there is any response from Gigabyte to AMD can it be published on
>> the list please?

> Good news everyone, Gigabyte have now implemented this feature, see
> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/




-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it

Re: IOMMU and AMD 890fx

[Jens Krehbiel-Gräther]

Hi all!

Asrock has released official 1.6 bios for the Asrock 890FX Deluxe3 
Board. There is iommu option in the bios, just flashed and checked. It 
works. So you can change the Wiki at

http://wiki.xensource.com/xenwiki/VTdHowTo

Asrock leaved beta status ;-)

Jens


Am 07.07.2010 09:47, schrieb Sander Eikelenboom:
> Hi all,
>
> I have received a beta-bios from MSI for the MSI 890FXA-GD70 that should support the IOMMU.
> Since I don't have this board (yet), if there is anyone who has it and cares to test, just drop an email and I can forward the bios.
>
> If it works, that would only leave Biostar not supporting it, and with Asrock, Gigabyte and MSI now in beta.
> --
>
> Sander
>
> Tuesday, July 6, 2010, 4:54:16 PM, you wrote:
>
>> On 10/06/10 19:49, Andrew Oakley wrote:
>>> I have a Gigabyte GA-890FXA-UD5 motherboard.  I have opened a support
>>> ticket with Gigabyte but have not yet received any reply.  The
>>> virtualization option in the BIOS is turned on, there do not appear to
>>> be any other relevant options.
>>>
>>> $ lspci -d 1002:5a23
>>> $ ls /sys/firmware/acpi/tables/
>>> APIC  DSDT  dynamic  FACP  FACS  HPET  MATS  MCFG  SSDT  TAMG
>>>
>>> If there is any response from Gigabyte to AMD can it be published on
>>> the list please?
>
>> Good news everyone, Gigabyte have now implemented this feature, see
>> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/
>
>
>
>

Re: IOMMU and AMD 890fx

[Sander Eikelenboom]

Done. :-)

Wednesday, July 7, 2010, 12:37:36 PM, you wrote:

> Hi all!

> Asrock has released official 1.6 bios for the Asrock 890FX Deluxe3 
> Board. There is iommu option in the bios, just flashed and checked. It 
> works. So you can change the Wiki at

> http://wiki.xensource.com/xenwiki/VTdHowTo

> Asrock leaved beta status ;-)

> Jens


> Am 07.07.2010 09:47, schrieb Sander Eikelenboom:
>> Hi all,
>>
>> I have received a beta-bios from MSI for the MSI 890FXA-GD70 that should support the IOMMU.
>> Since I don't have this board (yet), if there is anyone who has it and cares to test, just drop an email and I can forward the bios.
>>
>> If it works, that would only leave Biostar not supporting it, and with Asrock, Gigabyte and MSI now in beta.
>> --
>>
>> Sander
>>
>> Tuesday, July 6, 2010, 4:54:16 PM, you wrote:
>>
>>> On 10/06/10 19:49, Andrew Oakley wrote:
>>>> I have a Gigabyte GA-890FXA-UD5 motherboard.  I have opened a support
>>>> ticket with Gigabyte but have not yet received any reply.  The
>>>> virtualization option in the BIOS is turned on, there do not appear to
>>>> be any other relevant options.
>>>>
>>>> $ lspci -d 1002:5a23
>>>> $ ls /sys/firmware/acpi/tables/
>>>> APIC  DSDT  dynamic  FACP  FACS  HPET  MATS  MCFG  SSDT  TAMG
>>>>
>>>> If there is any response from Gigabyte to AMD can it be published on
>>>> the list please?
>>
>>> Good news everyone, Gigabyte have now implemented this feature, see
>>> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/
>>
>>
>>
>>



-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it

Re: IOMMU and AMD 890fx

[Sander Eikelenboom]

[-- Attachment #1: Type: text/plain, Size: 2280 bytes --]

Hello Jens,

Hi All,

With a bios flash, biostar does also seem to support the IOMMU.
So it seems all 890fx mobos have a (beta) bios available to support the IOMMU !

Reply from biostar tech support:

Yes, this motherboard supports IOMMU with 89FAD629.BST BIOS. Enable IOMMU
function via BIOS. I have to attached a picture for you reference. Also, the
IVRS table will auto create when enable IOMMU function via BIOS.

<< Enable IOMMU >>

=> Chipset / NorthBridge Configuration / IOMMU / Change from Disabled to
Enabled.

<< BIOS Download Link >>

=> http://www.biostar.com.tw/app/en/mb/bios.php?S_ID=482#


Wednesday, July 7, 2010, 12:37:36 PM, you wrote:

> Hi all!

> Asrock has released official 1.6 bios for the Asrock 890FX Deluxe3 
> Board. There is iommu option in the bios, just flashed and checked. It 
> works. So you can change the Wiki at

> http://wiki.xensource.com/xenwiki/VTdHowTo

> Asrock leaved beta status ;-)

> Jens


> Am 07.07.2010 09:47, schrieb Sander Eikelenboom:
>> Hi all,
>>
>> I have received a beta-bios from MSI for the MSI 890FXA-GD70 that should support the IOMMU.
>> Since I don't have this board (yet), if there is anyone who has it and cares to test, just drop an email and I can forward the bios.
>>
>> If it works, that would only leave Biostar not supporting it, and with Asrock, Gigabyte and MSI now in beta.
>> --
>>
>> Sander
>>
>> Tuesday, July 6, 2010, 4:54:16 PM, you wrote:
>>
>>> On 10/06/10 19:49, Andrew Oakley wrote:
>>>> I have a Gigabyte GA-890FXA-UD5 motherboard.  I have opened a support
>>>> ticket with Gigabyte but have not yet received any reply.  The
>>>> virtualization option in the BIOS is turned on, there do not appear to
>>>> be any other relevant options.
>>>>
>>>> $ lspci -d 1002:5a23
>>>> $ ls /sys/firmware/acpi/tables/
>>>> APIC  DSDT  dynamic  FACP  FACS  HPET  MATS  MCFG  SSDT  TAMG
>>>>
>>>> If there is any response from Gigabyte to AMD can it be published on
>>>> the list please?
>>
>>> Good news everyone, Gigabyte have now implemented this feature, see
>>> http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/
>>
>>
>>
>>



-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it

[-- Attachment #2: DSCN7637.jpg --]
[-- Type: image/jpeg, Size: 73355 bytes --]

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel