From mboxrd@z Thu Jan  1 00:00:00 1970
From: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Subject: Re: [PATCH 1/3] xen/vt-d: need barriers to workaround
 CLFLUSH
Date: Tue, 05 May 2015 11:46:17 -0400
Message-ID: <5548E5C9.7020108@oracle.com>
References: <1430705771-6744-1-git-send-email-tiejun.chen@intel.com>	<55474F790200007800076334@mail.emea.novell.com>
	<55473873.5030600@citrix.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xen.org>
In-Reply-To: <55473873.5030600@citrix.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Andrew Cooper <andrew.cooper3@citrix.com>, Jan Beulich <JBeulich@suse.com>, Tiejun Chen <tiejun.chen@intel.com>
Cc: yang.z.zhang@intel.com, jinsong.liu@alibaba-inc.com, kevin.tian@intel.com, keir@xen.org, xen-devel@lists.xen.org
List-Id: xen-devel@lists.xenproject.org

On 05/04/2015 05:14 AM, Andrew Cooper wrote:
> On 04/05/2015 09:52, Jan Beulich wrote:
>>>>> On 04.05.15 at 04:16, <tiejun.chen@intel.com> wrote:
>>> --- a/xen/drivers/passthrough/vtd/x86/vtd.c
>>> +++ b/xen/drivers/passthrough/vtd/x86/vtd.c
>>> @@ -56,7 +56,9 @@ unsigned int get_cache_line_size(void)
>>>   
>>>   void cacheline_flush(char * addr)
>>>   {
>>> +    mb();
>>>       clflush(addr);
>>> +    mb();
>>>   }
>> I think the purpose of the flush is to force write back, not to evict
>> the cache line, and if so wmb() would appear to be sufficient. As
>> the SDM says that's not the case, a comment explaining why wmb()
>> is not sufficient would seem necessary. Plus in the description I
>> think "serializing" needs to be changed to "fencing", as serialization
>> is not what we really care about here. If you and the maintainers
>> agree, I could certainly fix up both aspects while committing.
> On the subject of writebacks, we should get around to alternating-up the
> use of clflushopt and clwb, either of which would be better than a
> clflush in this case (avoiding the need for the leading mfence).
>
> However, the ISA extension document does not indicate which processors
> will have support for these new instructions.

https://software.intel.com/sites/default/files/managed/0d/53/319433-022.pdf

We really should add support for this. On shutting down a very large 
guest (hundreds of GB) we observed *minutes* spent in flushing IOMMU. 
This was due to serializing nature of CLFLUSH.

-boris