Re: [PATCH] cast used->idx to volatile

[Linhaifeng <haifeng.lin-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>]

On 2015/3/24 9:53, Xie, Huawei wrote:
> On 3/24/2015 9:00 AM, Linhaifeng wrote:
>>
>> On 2015/3/23 20:54, Xie, Huawei wrote:
>>>
>>>> -----Original Message-----
>>>> From: Linhaifeng [mailto:haifeng.lin-hv44wF8Li93QT0dZR+AlfA@public.gmane.org]
>>>> Sent: Monday, March 23, 2015 8:24 PM
>>>> To: dev-VfR2kkLFssw@public.gmane.org
>>>> Cc: Ouyang, Changchun; Xie, Huawei
>>>> Subject: Re: [dpdk-dev] [PATCH] cast used->idx to volatile
>>>>
>>>>
>>>>
>>>> On 2015/3/21 16:07, linhaifeng wrote:
>>>>> From: Linhaifeng <haifeng.lin-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>>>>>
>>>>> Same as rte_vhost_enqueue_burst we should cast used->idx
>>>>> to volatile before notify guest.
>>>>>
>>>>> Signed-off-by: Linhaifeng <haifeng.lin-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>>>>> ---
>>>>>  lib/librte_vhost/vhost_rxtx.c | 2 +-
>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/lib/librte_vhost/vhost_rxtx.c b/lib/librte_vhost/vhost_rxtx.c
>>>>> index 535c7a1..8d674d1 100644
>>>>> --- a/lib/librte_vhost/vhost_rxtx.c
>>>>> +++ b/lib/librte_vhost/vhost_rxtx.c
>>>>> @@ -722,7 +722,7 @@ rte_vhost_dequeue_burst(struct virtio_net *dev,
>>>> uint16_t queue_id,
>>>>>  	}
>>>>>
>>>>>  	rte_compiler_barrier();
>>>>> -	vq->used->idx += entry_success;
>>>>> +	*(volatile uint16_t *)&vq->used->idx += entry_success;
>>>
>>> Haifeng:
>>> We have compiler barrier before and an external function call behind, so we don't need volatile  here.
>>> Do you meet issue?
>>>
>> Tx_q is sometimes stopped when we use virtio_net. Because vhost thought there are no buffers in tx_q and virtio_net
>> though vhost haven't handle all packets so we have to restart VM to restore work.
>>
>> The status in VM is:
>> Mar 18 17:11:10 linux-b2ij kernel: [46337.246687] net eth7: virtnet_poll
>> Mar 18 17:11:10 linux-b2ij kernel: [46337.246690] net eth7: receive_buf
>> Mar 18 17:11:10 linux-b2ij kernel: [46337.246693] net eth7: vi->num=239
>> Mar 18 17:11:10 linux-b2ij kernel: [46337.246695] net eth7: svq:avail->idx=52939 used->idx=52939 num_free=18 num_added=0 svq->last_used_idx=52820
>> Mar 18 17:11:10 linux-b2ij kernel: [46337.246699] net eth7: rvq:avail->idx=36215 used->idx=35977 num_free=18 num_added=0 rvq->last_used_idx=35977
>> Mar 18 17:11:11 linux-b2ij kernel: [46337.901038] net eth7: dev_queue_xmit, qdisc->flags=4, qdisc->state deactiveed=0
>> Mar 18 17:11:12 linux-b2ij kernel: [46337.901042] net eth7: dev_queue_xmit, txq->state=1, stopped=1
>>
>> Why compiler barrier not take effect in our case? Is compiler barrier depended on -O3 option? We use -O2 option.
> compiler barrier always works regardless of the optimization option.
> I don't get your story, but the key thing is, do you check the asm code?
> If called from outside as an API, how is it possible it is optimized?
> there is only one update to used->idx in that function.


Do you mean rte_vhost_enqueue_burst also not need cast used->idx to volatile ? Why not remove it?

>>
>>>>>  	/* Kick guest if required. */
>>>>>  	if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT))
>>>>>  		eventfd_write((int)vq->callfd, 1);
>>>>>
>>>> --
>>>> Regards,
>>>> Haifeng
>>>
>>>
> 
> 
> .
> 

-- 
Regards,
Haifeng