On 06/11/2015 09:35 AM, Or Gerlitz wrote:
> Hi Doug,
> 
> This patchset adds the kernel control path for completion timestamping 
> support by user-space verbs consumers. 
> 
> Timestamping is used by applications in order to know when a WQE was 
> received/transmitted by the HW. The value is given is HCA hardware cycles,
> but could be easily converted as the hardware's core clock frequecny is 
> available through extension of query device. 
> 
> Moreover, we add an ability to read the HCA's current clock. This could be 
> useful on order to synchronize events to the wall clock.
> 
> This functionality is achieved by adding/extending the following verbs:
> 
> create_cq - create_cq is extended in order to allow passing creation flags
> to the CQ creation function. We change IB/core --> vendors API
> to be easily extendible by passing a struct which contains
> comp_vectors, cqe and the new flags parameter. In order to create
> CQ which supports timestamping, IB_CQ_FLAGS_TIMESTAMP_COMPLETION should be given.
> 
> query_device - We extend query_device uverb further by giving the hardware's
> clock frequency and the timestamp mask (the number of timestamp
> bits which are supported). If timestamp isn't supported, 0 is returned.
> 
> In order to read the timestamp in the WQE, the user needs to query the device 
> for support, create an appropriate CQ (using the extanded uverb with
> IB_CQ_FLAGS_TIMESTAMP_COMPLETION) and poll the CQ with an extended poll_cq verb (currently,
> only implemented in user-space).
> 
> In mlx4, allowing the user to read the core clock efficiently involves mapping
> this area of the hardware to user-space (being done by using a mmap command)
> and reading the clock from the correct offset of the page. 
> 
> This offset is returned in the vendor's specific data from mlx4's kernel driver 
> to the mlx4's user-space driver. query_device is modified in order to support
> passing this vendor specific data. A user-space application could use a new
> verb in order to read the hardware's clock.
> 
> Translating the hardware's clock into ms could be done by dividing this
> value by hca_core_clock (which is returned by the extended version of
> query_device uverb).
> 
> The below V2 --> V3 changes address the review comments on the kernel bits. 
> 
> Jason had another comment calling for re-thinking / questioning the 
> need for a dedicated uverbs_ex_cmd_mask in the IB device. This goes 
> beyond the scope of this specific series. 
> 
> Matan and Or.
> 
> Changes from V2:
>  (1) Use KHZ for hca_core_clock
>  (2) ib_create_cq gets const cq_attr
>  (3  use {} initialization instead of memset
>  (4) squashed last two mlx4 patches
> 
> Changes from V1:
>  (1) fixed lustre IB's code build
>  (2) squashed mlx4 V1 9-11 patches into one
>  (3) changed IB_CQ_FLAGS_TIMESTAMP --> IB_CQ_FLAGS_TIMESTAMP_COMPLETION
> 
> Changes from V0:
> (1) Pass ib_cq_init_attr instead of cqe and comp_vector.
> (2) Fix unneeded indentation.
> (3) Change flags to u32.
> (4) Add const to create_cq's ib_cq_init_attr argument in vendor implementation.

I've pulled this series in for 4.2.  However, I forgot to add the extra
reviewed-by tags before it was tagged, signed, and pushed to k.o.  My
apologies for that.