On 5/16/16 11:20 AM, Dagaen Golomb wrote:
> On Mon, May 16, 2016 at 12:11 PM, Dagaen Golomb <dgolomb@seas.upenn.edu> wrote:
>> On Mon, May 16, 2016 at 12:03 PM, Dagaen Golomb <dgolomb@seas.upenn.edu> wrote:
>>> On Mon, May 16, 2016 at 11:55 AM, Doug Goldstein <cardoe@cardoe.com> wrote:
>>>> On 5/15/16 8:41 PM, Dagaen Golomb wrote:
>>>>>> On 5/15/16 8:28 PM, Dagaen Golomb wrote:
>>>>>>>> On 5/15/16 11:40 AM, Dagaen Golomb wrote:
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> I'm having an interesting issue. I am working on a project that
>>>>>>>>> requires me to share memory between dom0 and domUs. I have this
>>>>>>>>> successfully working using the grant table and the XenStore to
>>>>>>>>> communicate grefs.
>>>>>>>>>
>>>>>>>>> My issue is this. I have one domU running Ubuntu 12.04 with a default
>>>>>>>>> 3.8.x kernel that has no issue reading or writing from the XenStore.
>>>>>>>>> My work also requires some kernel modifications, and we have made
>>>>>>>>> these changes in the 4.1.0 kernel. In particular, we've only added a
>>>>>>>>> simple hypercall. This modified kernel is what dom0 is running, on top
>>>>>>>>> of Xen 4.7 rc1.
>>>>>>>>
>>>>>>>> Without reading the rest of the thread but seeing the kernel versions.
>>>>>>>> Can you check how you're communicating to xenstore? Is it via
>>>>>>>> /dev/xen/xenbus or /proc/xen/xenbus? Anything after 3.14 will give you
>>>>>>>> deadlocks if you try to use /proc/xen/xenbus. Xen 4.6 and newer should
>>>>>>>> prefer /dev/xen/xenbus. Same thing can happen with privcmd but making
>>>>>>>> that default didn't land until Xen 4.7. Since you're on the right
>>>>>>>> versions I expect you're using /dev/xen/xenbus but you never know.
>>>>>>>
>>>>>>> How do I know which is being used? /dev/xen/xenbus is there and so is
>>>>>>> process/xen/xenbus. Could this be a problem with header version
>>>>>>> mismatches or something similar? I'm using the xen/xenstore.h header
>>>>>>> file for all of my xenstore interactions. I'm running Xen 4.7 so it
>>>>>>> should be in /dev/, and the old kernel is before 3.14 but the new one
>>>>>>> is after, but I would presume the standard headers are updated to
>>>>>>> account for this. Is there an easy way to check for this? Also, would
>>>>>>> the same issue cause writes to fails? Because writes from the same
>>>>>>> domain work fine, and appear to other domains using xenstore-ls.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Dagaen Golomb
>>>>>>>
>>>>>>
>>>>>> Use strace on the process and see what gets opened.
>>>>>
>>>>> Ah, of course. It seems both the working and non-working domains are
>>>>> using /proc/...
>>>>>
>>>>> Regards,
>>>>> Dagaen Golomb
>>>>>
>>>>
>>>>
>>>> How are you starting them? Can you confirm its attempting /dev/ first?
>>>> Xen 4.7 should prefer /dev/.
> 
> For all kernels in my domU, without setting any environment variables
> they use /proc/.
> For 4.1.0 this did not work, but works with /dev/ when using
> environment variable.
> Is this supposed to try /dev/ before /proc/? Because this doesn't
> appear the case according to strace.
> I think this is a bug.
> 
> Regards,
> Dagaen Golomb
> 

Is your domU using Xen 4.7 compiled utilities? or are you using distro
provided xenstore libraries?

-- 
Doug Goldstein