9p file system for xen

9p file system for xen

[Linda]

Hello,
      I worked this summer as an intern under Julien Grall and Wei Liu.  
My project was to develop a prototype/proof of concept xen front/back 
end for the 9p file system.  I mostly hacked the virtio 9p system.
     This project was not complete, at the end of the summer.  Julien 
said that you all wanted to include this in the next release of xen in 
January, and offered to take it over.  I told Julien I wanted to 
continue working on it, which I have been doing, very much in the 
background.
     I came upon a bug in my code recently that made me aware that I am 
not clear what the expectation for what I deliver should be: i.e., 
whether it's still a prototype, or whether this should be production 
software.
     Right now, I do not modify the toolstack (I never learned how), but 
rather start and pause my guest, and then modify xenstore, manually.   I 
can fix my bug in the same manner, but this will limit the usefulness of 
what I deliver.  To do more will hit up against the limitations of my 
time and knowledge.
     So please let me know what you're expecting, especially wrt the 
user interface, and when I would need to complete everything for this 
release.

Thank you.

Linda Jacobson

Re: 9p file system for xen

[Wei Liu]

Hi Linda

On Fri, Nov 13, 2015 at 10:23:22AM -0700, Linda wrote:
> Hello,
>      I worked this summer as an intern under Julien Grall and Wei Liu.  My
> project was to develop a prototype/proof of concept xen front/back end for
> the 9p file system.  I mostly hacked the virtio 9p system.
>     This project was not complete, at the end of the summer.  Julien said
> that you all wanted to include this in the next release of xen in January,
> and offered to take it over.  I told Julien I wanted to continue working on
> it, which I have been doing, very much in the background.
>     I came upon a bug in my code recently that made me aware that I am not
> clear what the expectation for what I deliver should be: i.e., whether it's
> still a prototype, or whether this should be production software.
>     Right now, I do not modify the toolstack (I never learned how), but
> rather start and pause my guest, and then modify xenstore, manually.   I can
> fix my bug in the same manner, but this will limit the usefulness of what I
> deliver.  To do more will hit up against the limitations of my time and
> knowledge.
>     So please let me know what you're expecting, especially wrt the user
> interface, and when I would need to complete everything for this release.
> 

If I interpret this correctly, you have a prototype that's working? Do
you have your code somewhere?

I think we would still like to include it in next release if possible --
that would require a properly implemented solution, not just a
prototype.  Let's assess the current situation and then decide what to
do next.

Wei.

> Thank you.
> 
> Linda Jacobson
> 
> 
> 
> 

Re: 9p file system for xen

[Linda]

Hi Wei,

On 11/16/2015 8:16 AM, Wei Liu wrote:
> Hi Linda
>
> On Fri, Nov 13, 2015 at 10:23:22AM -0700, Linda wrote:
>> Hello,
>>       I worked this summer as an intern under Julien Grall and Wei Liu.  My
>> project was to develop a prototype/proof of concept xen front/back end for
>> the 9p file system.  I mostly hacked the virtio 9p system.
>>      This project was not complete, at the end of the summer.  Julien said
>> that you all wanted to include this in the next release of xen in January,
>> and offered to take it over.  I told Julien I wanted to continue working on
>> it, which I have been doing, very much in the background.
>>      I came upon a bug in my code recently that made me aware that I am not
>> clear what the expectation for what I deliver should be: i.e., whether it's
>> still a prototype, or whether this should be production software.
>>      Right now, I do not modify the toolstack (I never learned how), but
>> rather start and pause my guest, and then modify xenstore, manually.   I can
>> fix my bug in the same manner, but this will limit the usefulness of what I
>> deliver.  To do more will hit up against the limitations of my time and
>> knowledge.
>>      So please let me know what you're expecting, especially wrt the user
>> interface, and when I would need to complete everything for this release.
>>
> If I interpret this correctly, you have a prototype that's working? Do
> you have your code somewhere?
No.  I hit a bug that I would fix differently, depending on my goal.
> I think we would still like to include it in next release if possible --
> that would require a properly implemented solution, not just a
> prototype.  Let's assess the current situation and then decide what to
The situation is, given my current knowledge and what my availability 
has been (it may improve), I can either:
     a.  Get a decent prototype working by the end of the year.  This 
would have certain values pre-written in xenstore, that I'm currently 
doing manually.  There are potentially some issues with mounting that I 
suspect need to be different for xen than they are for virtio - so 
either way, I need a clarification of how xen people want this to work.
     b.  Make sure what I've written is working, and pass it on to 
someone else to update the toolstack, and resolve the issues, described 
above.  In this scenario, I would need to know how much time that 
someone would need and just devote a week to getting this to them.

Either way, my next step is to sync up my qemu with the current qemu, 
and merge everything, and then my github will be correct, at which point 
you'll be able to access my most recent code.

Linda
> do next.
>
> Wei.
>
>> Thank you.
>>
>> Linda Jacobson
>>
>>
>>
>>

Re: 9p file system for xen

[Wei Liu]

On Mon, Nov 16, 2015 at 09:36:24AM -0700, Linda wrote:
> Hi Wei,
> 
> On 11/16/2015 8:16 AM, Wei Liu wrote:
> >Hi Linda
> >
> >On Fri, Nov 13, 2015 at 10:23:22AM -0700, Linda wrote:
> >>Hello,
> >>      I worked this summer as an intern under Julien Grall and Wei Liu.  My
> >>project was to develop a prototype/proof of concept xen front/back end for
> >>the 9p file system.  I mostly hacked the virtio 9p system.
> >>     This project was not complete, at the end of the summer.  Julien said
> >>that you all wanted to include this in the next release of xen in January,
> >>and offered to take it over.  I told Julien I wanted to continue working on
> >>it, which I have been doing, very much in the background.
> >>     I came upon a bug in my code recently that made me aware that I am not
> >>clear what the expectation for what I deliver should be: i.e., whether it's
> >>still a prototype, or whether this should be production software.
> >>     Right now, I do not modify the toolstack (I never learned how), but
> >>rather start and pause my guest, and then modify xenstore, manually.   I can
> >>fix my bug in the same manner, but this will limit the usefulness of what I
> >>deliver.  To do more will hit up against the limitations of my time and
> >>knowledge.
> >>     So please let me know what you're expecting, especially wrt the user
> >>interface, and when I would need to complete everything for this release.
> >>
> >If I interpret this correctly, you have a prototype that's working? Do
> >you have your code somewhere?
> No.  I hit a bug that I would fix differently, depending on my goal.
> >I think we would still like to include it in next release if possible --
> >that would require a properly implemented solution, not just a
> >prototype.  Let's assess the current situation and then decide what to
> The situation is, given my current knowledge and what my availability has
> been (it may improve), I can either:
>     a.  Get a decent prototype working by the end of the year.  This would
> have certain values pre-written in xenstore, that I'm currently doing
> manually.  There are potentially some issues with mounting that I suspect
> need to be different for xen than they are for virtio - so either way, I
> need a clarification of how xen people want this to work.
>     b.  Make sure what I've written is working, and pass it on to someone
> else to update the toolstack, and resolve the issues, described above.  In
> this scenario, I would need to know how much time that someone would need
> and just devote a week to getting this to them.
> 

Your description is too vague. I don't have clear idea what kind of bug
you encountered and what suggestion I can give.

The code freeze for next release is going to be end of March next year.
As software engineer often overestimates the progress he or she can
make, I would say we shall aim for getting something working as soon as
possible. Get the design straight and something clean by the end of this
year would be good.

> Either way, my next step is to sync up my qemu with the current qemu, and
> merge everything, and then my github will be correct, at which point you'll
> be able to access my most recent code.
> 

That would be a good first step. You don't actually need to fix the bug
for that if you don't know how to proceed yet.

Wei.

Re: 9p file system for xen

[Linda]

On 11/16/2015 9:51 AM, Wei Liu wrote:
> On Mon, Nov 16, 2015 at 09:36:24AM -0700, Linda wrote:
>> Hi Wei,
>>
>> On 11/16/2015 8:16 AM, Wei Liu wrote:
>>> Hi Linda
>>>
>>> On Fri, Nov 13, 2015 at 10:23:22AM -0700, Linda wrote:
>>>> Hello,
>>>>       I worked this summer as an intern under Julien Grall and Wei Liu.  My
>>>> project was to develop a prototype/proof of concept xen front/back end for
>>>> the 9p file system.  I mostly hacked the virtio 9p system.
>>>>      This project was not complete, at the end of the summer.  Julien said
>>>> that you all wanted to include this in the next release of xen in January,
>>>> and offered to take it over.  I told Julien I wanted to continue working on
>>>> it, which I have been doing, very much in the background.
>>>>      I came upon a bug in my code recently that made me aware that I am not
>>>> clear what the expectation for what I deliver should be: i.e., whether it's
>>>> still a prototype, or whether this should be production software.
>>>>      Right now, I do not modify the toolstack (I never learned how), but
>>>> rather start and pause my guest, and then modify xenstore, manually.   I can
>>>> fix my bug in the same manner, but this will limit the usefulness of what I
>>>> deliver.  To do more will hit up against the limitations of my time and
>>>> knowledge.
>>>>      So please let me know what you're expecting, especially wrt the user
>>>> interface, and when I would need to complete everything for this release.
>>>>
>>> If I interpret this correctly, you have a prototype that's working? Do
>>> you have your code somewhere?
>> No.  I hit a bug that I would fix differently, depending on my goal.
>>> I think we would still like to include it in next release if possible --
>>> that would require a properly implemented solution, not just a
>>> prototype.  Let's assess the current situation and then decide what to
>> The situation is, given my current knowledge and what my availability has
>> been (it may improve), I can either:
>>      a.  Get a decent prototype working by the end of the year.  This would
>> have certain values pre-written in xenstore, that I'm currently doing
>> manually.  There are potentially some issues with mounting that I suspect
>> need to be different for xen than they are for virtio - so either way, I
>> need a clarification of how xen people want this to work.
>>      b.  Make sure what I've written is working, and pass it on to someone
>> else to update the toolstack, and resolve the issues, described above.  In
>> this scenario, I would need to know how much time that someone would need
>> and just devote a week to getting this to them.
>>
> Your description is too vague. I don't have clear idea what kind of bug
> you encountered and what suggestion I can give.
The bug is a timing issue:  During virtio's probe step, on the front 
end, it initialized the mount path.  Since at that time, the front end 
doesn't have access to the back end's entries in xenstore (AFIACT), I 
either need to put it in xenstore prior to starting, or move the access 
to this information to later in the initialization.

Note, I used the past tense on what virtio did, as of last summer: when 
I looked at it last week, it appears to have changed since I first used 
it as a template.    I need to investigate this further.

Finally, I've made no provision for how to mount more than one file 
system for the same guest.  This is a feature that virtio provides for 
in the front-end code (as do I), but I am unclear about how this works 
in the back-end or at the user level.  This is what I suspect will be 
different in xen, and I'd like some input on what it should look like.
> The code freeze for next release is going to be end of March next year.
> As software engineer often overestimates the progress he or she can
> make, I would say we shall aim for getting something working as soon as
> possible. Get the design straight and something clean by the end of this
> year would be good.
Sounds good to me.  I'm happy to keep working on this.  I just didn't 
want to find myself in a position where I needed to pass this on to 
someone else, but I didn't give that person enough time to finish what 
I'd done.
>
>> Either way, my next step is to sync up my qemu with the current qemu, and
>> merge everything, and then my github will be correct, at which point you'll
>> be able to access my most recent code.
>>
> That would be a good first step. You don't actually need to fix the bug
> for that if you don't know how to proceed yet.
Good.  I'm glad we're on the same page, on this.

Linda
>
> Wei.
>

Re: 9p file system for xen

[Wei Liu]

On Mon, Nov 16, 2015 at 10:22:41AM -0700, Linda wrote:
> 
> 
> On 11/16/2015 9:51 AM, Wei Liu wrote:
> >On Mon, Nov 16, 2015 at 09:36:24AM -0700, Linda wrote:
> >>Hi Wei,
> >>
> >>On 11/16/2015 8:16 AM, Wei Liu wrote:
> >>>Hi Linda
> >>>
> >>>On Fri, Nov 13, 2015 at 10:23:22AM -0700, Linda wrote:
> >>>>Hello,
> >>>>      I worked this summer as an intern under Julien Grall and Wei Liu.  My
> >>>>project was to develop a prototype/proof of concept xen front/back end for
> >>>>the 9p file system.  I mostly hacked the virtio 9p system.
> >>>>     This project was not complete, at the end of the summer.  Julien said
> >>>>that you all wanted to include this in the next release of xen in January,
> >>>>and offered to take it over.  I told Julien I wanted to continue working on
> >>>>it, which I have been doing, very much in the background.
> >>>>     I came upon a bug in my code recently that made me aware that I am not
> >>>>clear what the expectation for what I deliver should be: i.e., whether it's
> >>>>still a prototype, or whether this should be production software.
> >>>>     Right now, I do not modify the toolstack (I never learned how), but
> >>>>rather start and pause my guest, and then modify xenstore, manually.   I can
> >>>>fix my bug in the same manner, but this will limit the usefulness of what I
> >>>>deliver.  To do more will hit up against the limitations of my time and
> >>>>knowledge.
> >>>>     So please let me know what you're expecting, especially wrt the user
> >>>>interface, and when I would need to complete everything for this release.
> >>>>
> >>>If I interpret this correctly, you have a prototype that's working? Do
> >>>you have your code somewhere?
> >>No.  I hit a bug that I would fix differently, depending on my goal.
> >>>I think we would still like to include it in next release if possible --
> >>>that would require a properly implemented solution, not just a
> >>>prototype.  Let's assess the current situation and then decide what to
> >>The situation is, given my current knowledge and what my availability has
> >>been (it may improve), I can either:
> >>     a.  Get a decent prototype working by the end of the year.  This would
> >>have certain values pre-written in xenstore, that I'm currently doing
> >>manually.  There are potentially some issues with mounting that I suspect
> >>need to be different for xen than they are for virtio - so either way, I
> >>need a clarification of how xen people want this to work.
> >>     b.  Make sure what I've written is working, and pass it on to someone
> >>else to update the toolstack, and resolve the issues, described above.  In
> >>this scenario, I would need to know how much time that someone would need
> >>and just devote a week to getting this to them.
> >>
> >Your description is too vague. I don't have clear idea what kind of bug
> >you encountered and what suggestion I can give.
> The bug is a timing issue:  During virtio's probe step, on the front end, it
> initialized the mount path.  Since at that time, the front end doesn't have
> access to the back end's entries in xenstore (AFIACT), I either need to put
> it in xenstore prior to starting, or move the access to this information to
> later in the initialization.
> 
> Note, I used the past tense on what virtio did, as of last summer: when I
> looked at it last week, it appears to have changed since I first used it as
> a template.    I need to investigate this further.
> 

OK.

> Finally, I've made no provision for how to mount more than one file system
> for the same guest.  This is a feature that virtio provides for in the
> front-end code (as do I), but I am unclear about how this works in the
> back-end or at the user level.  This is what I suspect will be different in
> xen, and I'd like some input on what it should look like.

I think this comes down to how your design the xenstore protocol to
represent different mount points.

> >The code freeze for next release is going to be end of March next year.
> >As software engineer often overestimates the progress he or she can
> >make, I would say we shall aim for getting something working as soon as
> >possible. Get the design straight and something clean by the end of this
> >year would be good.
> Sounds good to me.  I'm happy to keep working on this.  I just didn't want
> to find myself in a position where I needed to pass this on to someone else,
> but I didn't give that person enough time to finish what I'd done.

Depending on the situation, I can take over the code. You've done enough
for this project and we don't really want you to work on it for free --
we don't have provision for more funding at the moment.

If we end up taking over the project, we will still attribute the
initial implementation to you.

Wei.

Re: 9p file system for xen

[Linda]

Hi Wei,

On 11/16/2015 10:35 AM, Wei Liu wrote:
> On Mon, Nov 16, 2015 at 10:22:41AM -0700, Linda wrote:
...
>>
>> The bug is a timing issue:  During virtio's probe step, on the front end, it
>> initialized the mount path.  Since at that time, the front end doesn't have
>> access to the back end's entries in xenstore (AFIACT), I either need to put
>> it in xenstore prior to starting, or move the access to this information to
>> later in the initialization.
>>
>> Note, I used the past tense on what virtio did, as of last summer: when I
>> looked at it last week, it appears to have changed since I first used it as
>> a template.    I need to investigate this further.
>>
> OK.
>
>> Finally, I've made no provision for how to mount more than one file system
>> for the same guest.  This is a feature that virtio provides for in the
>> front-end code (as do I), but I am unclear about how this works in the
>> back-end or at the user level.  This is what I suspect will be different in
>> xen, and I'd like some input on what it should look like.
> I think this comes down to how your design the xenstore protocol to
> represent different mount points.
And just reading this gave me the answer I need.
>
>>> The code freeze for next release is going to be end of March next year.
>>> As software engineer often overestimates the progress he or she can
>>> make, I would say we shall aim for getting something working as soon as
>>> possible. Get the design straight and something clean by the end of this
>>> year would be good.
>> Sounds good to me.  I'm happy to keep working on this.  I just didn't want
>> to find myself in a position where I needed to pass this on to someone else,
>> but I didn't give that person enough time to finish what I'd done.
> Depending on the situation, I can take over the code. You've done enough
> for this project and we don't really want you to work on it for free --
> we don't have provision for more funding at the moment.
Understood.
> If we end up taking over the project, we will still attribute the
> initial implementation to you.
     Thanks.  Julien said essentially the same thing.  Right now, I'm 
working on average, less than 10 hours/week, so it's enough to keep my 
mind engaged, but it doesn't interfere with anything else.
     I will be working for pay, in some capacity (TBD), after the first 
of the year.   Right now, I'm working to line things up.
     Unless something changes drastically, I'll continue to work on this 
until the end of the year.  I'll start by cleaning things up, and keep 
it that way, so no matter what happens, you, or Julien, can take it over.

Linda
>
> Wei.
>

Re: 9p file system for xen

[Neil Sikka]

[-- Attachment #1.1: Type: text/plain, Size: 3479 bytes --]

How does Linda's work relate to Wei's patches available here (I didnt see
them in Xen-4.6.0):

http://downloads.xen.org/Wiki/VirtioOnXen/qemu-01-xenpv-exec.patch
http://downloads.xen.org/Wiki/VirtioOnXen/qemu-02-virtio-for-pv.patch

Also, since 9p is being worked on, which is a filesystem that should be
implemented in a kernel rather than a hypervisor, are you looking to
contribute this driver to the Linux kernel?

On Mon, Nov 16, 2015 at 10:02 PM, Linda <lindaj@jma3.com> wrote:

> Hi Wei,
>
> On 11/16/2015 10:35 AM, Wei Liu wrote:
>
>> On Mon, Nov 16, 2015 at 10:22:41AM -0700, Linda wrote:
>>
> ...
>
>>
>>> The bug is a timing issue:  During virtio's probe step, on the front
>>> end, it
>>> initialized the mount path.  Since at that time, the front end doesn't
>>> have
>>> access to the back end's entries in xenstore (AFIACT), I either need to
>>> put
>>> it in xenstore prior to starting, or move the access to this information
>>> to
>>> later in the initialization.
>>>
>>> Note, I used the past tense on what virtio did, as of last summer: when I
>>> looked at it last week, it appears to have changed since I first used it
>>> as
>>> a template.    I need to investigate this further.
>>>
>>> OK.
>>
>> Finally, I've made no provision for how to mount more than one file system
>>> for the same guest.  This is a feature that virtio provides for in the
>>> front-end code (as do I), but I am unclear about how this works in the
>>> back-end or at the user level.  This is what I suspect will be different
>>> in
>>> xen, and I'd like some input on what it should look like.
>>>
>> I think this comes down to how your design the xenstore protocol to
>> represent different mount points.
>>
> And just reading this gave me the answer I need.
>
>>
>> The code freeze for next release is going to be end of March next year.
>>>> As software engineer often overestimates the progress he or she can
>>>> make, I would say we shall aim for getting something working as soon as
>>>> possible. Get the design straight and something clean by the end of this
>>>> year would be good.
>>>>
>>> Sounds good to me.  I'm happy to keep working on this.  I just didn't
>>> want
>>> to find myself in a position where I needed to pass this on to someone
>>> else,
>>> but I didn't give that person enough time to finish what I'd done.
>>>
>> Depending on the situation, I can take over the code. You've done enough
>> for this project and we don't really want you to work on it for free --
>> we don't have provision for more funding at the moment.
>>
> Understood.
>
>> If we end up taking over the project, we will still attribute the
>> initial implementation to you.
>>
>     Thanks.  Julien said essentially the same thing.  Right now, I'm
> working on average, less than 10 hours/week, so it's enough to keep my mind
> engaged, but it doesn't interfere with anything else.
>     I will be working for pay, in some capacity (TBD), after the first of
> the year.   Right now, I'm working to line things up.
>     Unless something changes drastically, I'll continue to work on this
> until the end of the year.  I'll start by cleaning things up, and keep it
> that way, so no matter what happens, you, or Julien, can take it over.
>
> Linda
>
>
>> Wei.
>>
>>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>



-- 
My Blog: http://www.neilscomputerblog.blogspot.com/
Twitter: @neilsikka

[-- Attachment #1.2: Type: text/html, Size: 5645 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 9p file system for xen

[Linda]

[-- Attachment #1.1: Type: text/plain, Size: 4537 bytes --]



On 11/17/2015 11:35 AM, Neil Sikka wrote:
> How does Linda's work relate to Wei's patches available here (I didnt 
> see them in Xen-4.6.0):
>
> http://downloads.xen.org/Wiki/VirtioOnXen/qemu-01-xenpv-exec.patch
> http://downloads.xen.org/Wiki/VirtioOnXen/qemu-02-virtio-for-pv.patch
I'll let Wei answer this.
>
> Also, since 9p is being worked on, which is a filesystem that should 
> be implemented in a kernel rather than a hypervisor, are you looking 
> to contribute this driver to the Linux kernel?
What I did was write new kernel routines and new Qemu routines, as well 
as modifying a few existing Qemu files.  The initialization is currently 
done manually by modifying xenstore.  This is the only code that 
properly belongs in the hypervisor.

I hope this clarifies things.

Linda
>
> On Mon, Nov 16, 2015 at 10:02 PM, Linda <lindaj@jma3.com 
> <mailto:lindaj@jma3.com>> wrote:
>
>     Hi Wei,
>
>     On 11/16/2015 10:35 AM, Wei Liu wrote:
>
>         On Mon, Nov 16, 2015 at 10:22:41AM -0700, Linda wrote:
>
>     ...
>
>
>             The bug is a timing issue:  During virtio's probe step, on
>             the front end, it
>             initialized the mount path.  Since at that time, the front
>             end doesn't have
>             access to the back end's entries in xenstore (AFIACT), I
>             either need to put
>             it in xenstore prior to starting, or move the access to
>             this information to
>             later in the initialization.
>
>             Note, I used the past tense on what virtio did, as of last
>             summer: when I
>             looked at it last week, it appears to have changed since I
>             first used it as
>             a template.    I need to investigate this further.
>
>         OK.
>
>             Finally, I've made no provision for how to mount more than
>             one file system
>             for the same guest.  This is a feature that virtio
>             provides for in the
>             front-end code (as do I), but I am unclear about how this
>             works in the
>             back-end or at the user level.  This is what I suspect
>             will be different in
>             xen, and I'd like some input on what it should look like.
>
>         I think this comes down to how your design the xenstore
>         protocol to
>         represent different mount points.
>
>     And just reading this gave me the answer I need.
>
>
>                 The code freeze for next release is going to be end of
>                 March next year.
>                 As software engineer often overestimates the progress
>                 he or she can
>                 make, I would say we shall aim for getting something
>                 working as soon as
>                 possible. Get the design straight and something clean
>                 by the end of this
>                 year would be good.
>
>             Sounds good to me.  I'm happy to keep working on this.  I
>             just didn't want
>             to find myself in a position where I needed to pass this
>             on to someone else,
>             but I didn't give that person enough time to finish what
>             I'd done.
>
>         Depending on the situation, I can take over the code. You've
>         done enough
>         for this project and we don't really want you to work on it
>         for free --
>         we don't have provision for more funding at the moment.
>
>     Understood.
>
>         If we end up taking over the project, we will still attribute the
>         initial implementation to you.
>
>         Thanks.  Julien said essentially the same thing.  Right now,
>     I'm working on average, less than 10 hours/week, so it's enough to
>     keep my mind engaged, but it doesn't interfere with anything else.
>         I will be working for pay, in some capacity (TBD), after the
>     first of the year.   Right now, I'm working to line things up.
>         Unless something changes drastically, I'll continue to work on
>     this until the end of the year.  I'll start by cleaning things up,
>     and keep it that way, so no matter what happens, you, or Julien,
>     can take it over.
>
>     Linda
>
>
>         Wei.
>
>
>
>     _______________________________________________
>     Xen-devel mailing list
>     Xen-devel@lists.xen.org <mailto:Xen-devel@lists.xen.org>
>     http://lists.xen.org/xen-devel
>
>
>
>
> -- 
> My Blog: http://www.neilscomputerblog.blogspot.com/
> Twitter: @neilsikka


[-- Attachment #1.2: Type: text/html, Size: 9270 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 9p file system for xen

[Wei Liu]

On Tue, Nov 17, 2015 at 12:50:29PM -0700, Linda wrote:
> 
> 
> On 11/17/2015 11:35 AM, Neil Sikka wrote:
> >How does Linda's work relate to Wei's patches available here (I didnt see
> >them in Xen-4.6.0):
> >
> >http://downloads.xen.org/Wiki/VirtioOnXen/qemu-01-xenpv-exec.patch
> >http://downloads.xen.org/Wiki/VirtioOnXen/qemu-02-virtio-for-pv.patch
> I'll let Wei answer this.

That wasn't upstreamed at all. Admittedly that was done when I didn't
know much about Xen. I would have done that project differently
nowadays.

And to clarify things: virtio on Xen is a different project than 9pfs on
Xen. 9pfs is not tied to virtio in any way. Just that there is currently
only virtio-9pfs available.

So to make 9pfs work on Xen, there are at least two ways. One is to make
virtio work on Xen so that we subsequently get virtio-9pfs (along with
all other virtio devices); the other is to implement xen-9pfs (I made up
this name).

I'm not sure whether you're interested in virtio on Xen or just 9pfs.

> >
> >Also, since 9p is being worked on, which is a filesystem that should be
> >implemented in a kernel rather than a hypervisor, are you looking to
> >contribute this driver to the Linux kernel?
> What I did was write new kernel routines and new Qemu routines, as well as

I think Neil was talking about the "new kernel routines". Yes, that
would need to be upstreamed eventually.

Wei.

> modifying a few existing Qemu files.  The initialization is currently done
> manually by modifying xenstore.  This is the only code that properly belongs
> in the hypervisor.
> 
> I hope this clarifies things.
> 
> Linda

Re: 9p file system for xen

[Neil Sikka]

[-- Attachment #1.1: Type: text/plain, Size: 2148 bytes --]

Is there any documentation about planned interfaces and API contracts for
people building around the virtio/9pfs layers? For example, while this is
still getting debugged/checked in, in order to build DomU support for these
devices, the expected API contracts/interfaces would need to be known.

On Wed, Nov 18, 2015 at 4:56 AM, Wei Liu <wei.liu2@citrix.com> wrote:

> On Tue, Nov 17, 2015 at 12:50:29PM -0700, Linda wrote:
> >
> >
> > On 11/17/2015 11:35 AM, Neil Sikka wrote:
> > >How does Linda's work relate to Wei's patches available here (I didnt
> see
> > >them in Xen-4.6.0):
> > >
> > >http://downloads.xen.org/Wiki/VirtioOnXen/qemu-01-xenpv-exec.patch
> > >http://downloads.xen.org/Wiki/VirtioOnXen/qemu-02-virtio-for-pv.patch
> > I'll let Wei answer this.
>
> That wasn't upstreamed at all. Admittedly that was done when I didn't
> know much about Xen. I would have done that project differently
> nowadays.
>
> And to clarify things: virtio on Xen is a different project than 9pfs on
> Xen. 9pfs is not tied to virtio in any way. Just that there is currently
> only virtio-9pfs available.
>
> So to make 9pfs work on Xen, there are at least two ways. One is to make
> virtio work on Xen so that we subsequently get virtio-9pfs (along with
> all other virtio devices); the other is to implement xen-9pfs (I made up
> this name).
>
> I'm not sure whether you're interested in virtio on Xen or just 9pfs.
>
> > >
> > >Also, since 9p is being worked on, which is a filesystem that should be
> > >implemented in a kernel rather than a hypervisor, are you looking to
> > >contribute this driver to the Linux kernel?
> > What I did was write new kernel routines and new Qemu routines, as well
> as
>
> I think Neil was talking about the "new kernel routines". Yes, that
> would need to be upstreamed eventually.
>
> Wei.
>
> > modifying a few existing Qemu files.  The initialization is currently
> done
> > manually by modifying xenstore.  This is the only code that properly
> belongs
> > in the hypervisor.
> >
> > I hope this clarifies things.
> >
> > Linda
>



-- 
My Blog: http://www.neilscomputerblog.blogspot.com/
Twitter: @neilsikka

[-- Attachment #1.2: Type: text/html, Size: 3236 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 9p file system for xen

[Wei Liu]

On Thu, Nov 19, 2015 at 09:55:00AM -0500, Neil Sikka wrote:
> Is there any documentation about planned interfaces and API contracts for
> people building around the virtio/9pfs layers? For example, while this is

I assume that you're interested in getting 9pfs to work but don't care
much about how it is made to work? I ask because I'm a bit confused by
the notion of "virtio/9pfs" because what Linda did wasn't based on
virtio transport.

> still getting debugged/checked in, in order to build DomU support for these
> devices, the expected API contracts/interfaces would need to be known.
> 

I'm not sure I get your question. What do you mean by "build DomU
support"? What kind of DomU support needs insider knowledge of how 9pfs
is setup? Are you talking about your own management stack that starts
VM?

Wei.

Re: 9p file system for xen

[Wei Liu]

On Mon, Nov 16, 2015 at 08:02:35PM -0700, Linda wrote:
> Hi Wei,
> 
> On 11/16/2015 10:35 AM, Wei Liu wrote:
> >On Mon, Nov 16, 2015 at 10:22:41AM -0700, Linda wrote:
> ...
> >>
> >>The bug is a timing issue:  During virtio's probe step, on the front end, it
> >>initialized the mount path.  Since at that time, the front end doesn't have
> >>access to the back end's entries in xenstore (AFIACT), I either need to put
> >>it in xenstore prior to starting, or move the access to this information to
> >>later in the initialization.
> >>
> >>Note, I used the past tense on what virtio did, as of last summer: when I
> >>looked at it last week, it appears to have changed since I first used it as
> >>a template.    I need to investigate this further.
> >>
> >OK.
> >
> >>Finally, I've made no provision for how to mount more than one file system
> >>for the same guest.  This is a feature that virtio provides for in the
> >>front-end code (as do I), but I am unclear about how this works in the
> >>back-end or at the user level.  This is what I suspect will be different in
> >>xen, and I'd like some input on what it should look like.
> >I think this comes down to how your design the xenstore protocol to
> >represent different mount points.
> And just reading this gave me the answer I need.
> >
> >>>The code freeze for next release is going to be end of March next year.
> >>>As software engineer often overestimates the progress he or she can
> >>>make, I would say we shall aim for getting something working as soon as
> >>>possible. Get the design straight and something clean by the end of this
> >>>year would be good.
> >>Sounds good to me.  I'm happy to keep working on this.  I just didn't want
> >>to find myself in a position where I needed to pass this on to someone else,
> >>but I didn't give that person enough time to finish what I'd done.
> >Depending on the situation, I can take over the code. You've done enough
> >for this project and we don't really want you to work on it for free --
> >we don't have provision for more funding at the moment.
> Understood.
> >If we end up taking over the project, we will still attribute the
> >initial implementation to you.
>     Thanks.  Julien said essentially the same thing.  Right now, I'm working
> on average, less than 10 hours/week, so it's enough to keep my mind engaged,
> but it doesn't interfere with anything else.
>     I will be working for pay, in some capacity (TBD), after the first of
> the year.   Right now, I'm working to line things up.
>     Unless something changes drastically, I'll continue to work on this
> until the end of the year.  I'll start by cleaning things up, and keep it
> that way, so no matter what happens, you, or Julien, can take it over.
> 

Linda, I will be on vacation early December until early January. I do
want to be do some preparatory work before I leave. Can you just post
what you have to xen-devel? Your code doesn't need to be perfect. I can
handle the rest.

Wei.

> Linda
> >
> >Wei.
> >
> 

Re: 9p file system for xen

[Linda]

Hi Wei,

On 11/19/2015 8:03 AM, Wei Liu wrote:
> On Thu, Nov 19, 2015 at 09:55:00AM -0500, Neil Sikka wrote:
>> Is there any documentation about planned interfaces and API contracts for
>> people building around the virtio/9pfs layers? For example, while this is
> I assume that you're interested in getting 9pfs to work but don't care
> much about how it is made to work? I ask because I'm a bit confused by
> the notion of "virtio/9pfs" because what Linda did wasn't based on
> virtio transport.
A clarification:  If you mean the virtio 9pfs transport, my code is 
based on that, and sits on top of it.
>
>> still getting debugged/checked in, in order to build DomU support for these
>> devices, the expected API contracts/interfaces would need to be known.
>>
> I'm not sure I get your question. What do you mean by "build DomU
> support"? What kind of DomU support needs insider knowledge of how 9pfs
> is setup? Are you talking about your own management stack that starts
> VM?
>
> Wei.
>

Re: 9p file system for xen

[Neil Sikka]

[-- Attachment #1.1: Type: text/plain, Size: 1581 bytes --]

Wei, based on our discussions, you will review as much as possible before
leaving for vacation, and have started last Wednesday. Could you please
share what you are reviewing so I (and anyone else interested) can also
assist in reviewing it?

As you said, the front/back end negotiation protocols need to be designed
and agreed upon, and we can start that now in parallel with the code review.

On Thu, Nov 19, 2015 at 11:23 AM, Linda <lindaj@jma3.com> wrote:

> Hi Wei,
>
> On 11/19/2015 8:03 AM, Wei Liu wrote:
>
>> On Thu, Nov 19, 2015 at 09:55:00AM -0500, Neil Sikka wrote:
>>
>>> Is there any documentation about planned interfaces and API contracts for
>>> people building around the virtio/9pfs layers? For example, while this is
>>>
>> I assume that you're interested in getting 9pfs to work but don't care
>> much about how it is made to work? I ask because I'm a bit confused by
>> the notion of "virtio/9pfs" because what Linda did wasn't based on
>> virtio transport.
>>
> A clarification:  If you mean the virtio 9pfs transport, my code is based
> on that, and sits on top of it.
>
>
>> still getting debugged/checked in, in order to build DomU support for
>>> these
>>> devices, the expected API contracts/interfaces would need to be known.
>>>
>>> I'm not sure I get your question. What do you mean by "build DomU
>> support"? What kind of DomU support needs insider knowledge of how 9pfs
>> is setup? Are you talking about your own management stack that starts
>> VM?
>>
>> Wei.
>>
>>
>


-- 
My Blog: http://www.neilscomputerblog.blogspot.com/
Twitter: @neilsikka

[-- Attachment #1.2: Type: text/html, Size: 2911 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 9p file system for xen

[Wei Liu]

On Mon, Nov 23, 2015 at 10:51:04AM -0500, Neil Sikka wrote:
> Wei, based on our discussions, you will review as much as possible before
> leaving for vacation, and have started last Wednesday. Could you please
> share what you are reviewing so I (and anyone else interested) can also
> assist in reviewing it?
> 

What I meant was "when patches are posted to xen-devel I will try to
review as much as possible". There is no patch on xen-devel so I haven't
actually reviewed anything, nor can anyone else who is interested in
getting it work review anything.

What I started last Wednesday was refactoring some QEMU code (only
renaming files and making QEMU build), so that QEMU 9pfs doesn't just
assume it is using virtio transport. I didn't mean I started review
anything.

Further discussion should go to xen-devel. There is definitely a need to
get input from wider community to better coordinate this work.

Wei.

> As you said, the front/back end negotiation protocols need to be designed
> and agreed upon, and we can start that now in parallel with the code review.
> 
> On Thu, Nov 19, 2015 at 11:23 AM, Linda <lindaj@jma3.com> wrote:
> 
> > Hi Wei,
> >
> > On 11/19/2015 8:03 AM, Wei Liu wrote:
> >
> >> On Thu, Nov 19, 2015 at 09:55:00AM -0500, Neil Sikka wrote:
> >>
> >>> Is there any documentation about planned interfaces and API contracts for
> >>> people building around the virtio/9pfs layers? For example, while this is
> >>>
> >> I assume that you're interested in getting 9pfs to work but don't care
> >> much about how it is made to work? I ask because I'm a bit confused by
> >> the notion of "virtio/9pfs" because what Linda did wasn't based on
> >> virtio transport.
> >>
> > A clarification:  If you mean the virtio 9pfs transport, my code is based
> > on that, and sits on top of it.
> >
> >
> >> still getting debugged/checked in, in order to build DomU support for
> >>> these
> >>> devices, the expected API contracts/interfaces would need to be known.
> >>>
> >>> I'm not sure I get your question. What do you mean by "build DomU
> >> support"? What kind of DomU support needs insider knowledge of how 9pfs
> >> is setup? Are you talking about your own management stack that starts
> >> VM?
> >>
> >> Wei.
> >>
> >>
> >
> 
> 
> -- 
> My Blog: http://www.neilscomputerblog.blogspot.com/
> Twitter: @neilsikka

Re: 9p file system for xen

[Neil Sikka]

[-- Attachment #1.1: Type: text/plain, Size: 2129 bytes --]

Hi Wei, could you please explain why/how you would have done the project
differently now and why these patches are not "good"? From my conversation
with Linda, I understood that her code is "Independent of virtio except the
9pvirtio specific code, which is used extensively."

On Wed, Nov 18, 2015 at 4:56 AM, Wei Liu <wei.liu2@citrix.com> wrote:

> On Tue, Nov 17, 2015 at 12:50:29PM -0700, Linda wrote:
> >
> >
> > On 11/17/2015 11:35 AM, Neil Sikka wrote:
> > >How does Linda's work relate to Wei's patches available here (I didnt
> see
> > >them in Xen-4.6.0):
> > >
> > >http://downloads.xen.org/Wiki/VirtioOnXen/qemu-01-xenpv-exec.patch
> > >http://downloads.xen.org/Wiki/VirtioOnXen/qemu-02-virtio-for-pv.patch
> > I'll let Wei answer this.
>
> That wasn't upstreamed at all. Admittedly that was done when I didn't
> know much about Xen. I would have done that project differently
> nowadays.
>
> And to clarify things: virtio on Xen is a different project than 9pfs on
> Xen. 9pfs is not tied to virtio in any way. Just that there is currently
> only virtio-9pfs available.
>
> So to make 9pfs work on Xen, there are at least two ways. One is to make
> virtio work on Xen so that we subsequently get virtio-9pfs (along with
> all other virtio devices); the other is to implement xen-9pfs (I made up
> this name).
>
> I'm not sure whether you're interested in virtio on Xen or just 9pfs.
>
> > >
> > >Also, since 9p is being worked on, which is a filesystem that should be
> > >implemented in a kernel rather than a hypervisor, are you looking to
> > >contribute this driver to the Linux kernel?
> > What I did was write new kernel routines and new Qemu routines, as well
> as
>
> I think Neil was talking about the "new kernel routines". Yes, that
> would need to be upstreamed eventually.
>
> Wei.
>
> > modifying a few existing Qemu files.  The initialization is currently
> done
> > manually by modifying xenstore.  This is the only code that properly
> belongs
> > in the hypervisor.
> >
> > I hope this clarifies things.
> >
> > Linda
>



-- 
My Blog: http://www.neilscomputerblog.blogspot.com/
Twitter: @neilsikka

[-- Attachment #1.2: Type: text/html, Size: 3218 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: 9p file system for xen

[Wei Liu]

On Mon, Nov 30, 2015 at 12:19:18PM -0500, Neil Sikka wrote:
> Hi Wei, could you please explain why/how you would have done the project
> differently now and why these patches are not "good"? From my conversation
> with Linda, I understood that her code is "Independent of virtio except the
> 9pvirtio specific code, which is used extensively."
> 

I need to implement a xen transport for 9pfs. Linda was essentially
doing the same. But she didn't specify the canonical protocol between
frontend and backend.

As for "9pvirtio specific code", I think there is misunderstanding
because though a lot of files in QEMU are prefixed with virtio they are
actually not specific to virtio at all. I think the "independent of
virtio ..." part was referring to the new transport she wrote.

Wei.

Re: 9p file system for xen

[Linda]

On 12/1/2015 4:47 AM, Wei Liu wrote:
> On Mon, Nov 30, 2015 at 12:19:18PM -0500, Neil Sikka wrote:
>> Hi Wei, could you please explain why/how you would have done the project
>> differently now and why these patches are not "good"? From my conversation
>> with Linda, I understood that her code is "Independent of virtio except the
>> 9pvirtio specific code, which is used extensively."
>>
> I need to implement a xen transport for 9pfs. Linda was essentially
> doing the same. But she didn't specify the canonical protocol between
> frontend and backend.
For my own edification:  In the interests of the limited time of my 
internship, we decided I shouldn't do the initialization using the xen 
toolstack.  Were there are other expediencies that I'm unaware of?

I tried to follow the xen handshaking protocol between front and back 
end at startup.

Thanks.

Linda
>
> As for "9pvirtio specific code", I think there is misunderstanding
> because though a lot of files in QEMU are prefixed with virtio they are
> actually not specific to virtio at all. I think the "independent of
> virtio ..." part was referring to the new transport she wrote.
>
> Wei.
>

Re: 9p file system for xen

[Wei Liu]

On Tue, Dec 01, 2015 at 07:37:43AM -0700, Linda wrote:
> 
> 
> On 12/1/2015 4:47 AM, Wei Liu wrote:
> >On Mon, Nov 30, 2015 at 12:19:18PM -0500, Neil Sikka wrote:
> >>Hi Wei, could you please explain why/how you would have done the project
> >>differently now and why these patches are not "good"? From my conversation
> >>with Linda, I understood that her code is "Independent of virtio except the
> >>9pvirtio specific code, which is used extensively."
> >>
> >I need to implement a xen transport for 9pfs. Linda was essentially
> >doing the same. But she didn't specify the canonical protocol between
> >frontend and backend.
> For my own edification:  In the interests of the limited time of my
> internship, we decided I shouldn't do the initialization using the xen
> toolstack.  Were there are other expediencies that I'm unaware of?
> 

It's not about toolstack. Toolstack merely sets up xenstore nodes
according to the protocol.

> I tried to follow the xen handshaking protocol between front and back end at
> startup.
> 

Yes, that's the right direction. Following existing convention is good
enough for an intern project. Specifying the protocol in detailed is not
the requirement for a prototype.

But in the end to upstream xen-9pfs a canonical protocol is required.  A
blessed version of protocol needs to be committed to xen.git tree.  We
have a bunch of those in xen.git/xen/include/public/io/ directory.

Wei.

> Thanks.
> 
> Linda
> >
> >As for "9pvirtio specific code", I think there is misunderstanding
> >because though a lot of files in QEMU are prefixed with virtio they are
> >actually not specific to virtio at all. I think the "independent of
> >virtio ..." part was referring to the new transport she wrote.
> >
> >Wei.
> >
> 

Re: 9p file system for xen

[Linda]

On 12/1/2015 7:46 AM, Wei Liu wrote:
> On Tue, Dec 01, 2015 at 07:37:43AM -0700, Linda wrote:
>>
>> On 12/1/2015 4:47 AM, Wei Liu wrote:
>>> On Mon, Nov 30, 2015 at 12:19:18PM -0500, Neil Sikka wrote:
>>>> Hi Wei, could you please explain why/how you would have done the project
>>>> differently now and why these patches are not "good"? From my conversation
>>>> with Linda, I understood that her code is "Independent of virtio except the
>>>> 9pvirtio specific code, which is used extensively."
>>>>
>>> I need to implement a xen transport for 9pfs. Linda was essentially
>>> doing the same. But she didn't specify the canonical protocol between
>>> frontend and backend.
>> For my own edification:  In the interests of the limited time of my
>> internship, we decided I shouldn't do the initialization using the xen
>> toolstack.  Were there are other expediencies that I'm unaware of?
>>
> It's not about toolstack. Toolstack merely sets up xenstore nodes
> according to the protocol.
>
>> I tried to follow the xen handshaking protocol between front and back end at
>> startup.
>>
> Yes, that's the right direction. Following existing convention is good
> enough for an intern project. Specifying the protocol in detailed is not
> the requirement for a prototype.
>
> But in the end to upstream xen-9pfs a canonical protocol is required.  A
> blessed version of protocol needs to be committed to xen.git tree.  We
> have a bunch of those in xen.git/xen/include/public/io/ directory.
>
> Wei.
Thank you.

L
>> Thanks.
>>
>> Linda
>>> As for "9pvirtio specific code", I think there is misunderstanding
>>> because though a lot of files in QEMU are prefixed with virtio they are
>>> actually not specific to virtio at all. I think the "independent of
>>> virtio ..." part was referring to the new transport she wrote.
>>>
>>> Wei.
>>>

Re: 9p file system for xen

[Linda]

[-- Attachment #1: Type: text/plain, Size: 889 bytes --]

Hello,
     Wei has suggested I "send a summarised email on [my] work to 
xen-devel so that people are aware of [my] work."
     To that end, and to save Wei from reinventing the wheel, I am 
providing the following:

      First to correct a misconception, I followed the state change 
protocol of xen, as outlined by Ian Campbell in his 6/11 email on 
"Re:[xen-devel]grant tables and driver handshaking".
     This was tested early in the summer when I passed a small amount of 
data between my initial front and back ends.   The code resides in the 
files p9_front.c, p9_front_driver.c on the front end and xen_9pif.c on 
the backend.
     I am attaching an updated version of a writeup I made this summer.  
It's table of contents is:
     1.  The state of the project
     2.  The approach I took
     3.  How to build my project
     4.  How to use it.

Sincerely,

Linda Jacobson



[-- Attachment #2: ljoutreach.txt --]
[-- Type: text/plain, Size: 8964 bytes --]

I.  PROJECT STATUS

The project compiles and builds.  The backend initialization works.  The front
end works partway through initialization. It dies in probe.  This is because I
initialized the mount path in the backend using the xenstore backend path, and
tried to access it in the frontend with its path.  Currently this path is hardcoded
in the init code in xen_9pif.c.  As I wrote in my commit message, this should be
done in initialization.  Ideally, to handle multiple mounts, the xenstore path
should have a "mount directory" that contains all the file system mount paths.

In addition, all of the front and backend code to manage the ring and the
one entry grant table, as well as transferring data to and from the shared page
has been tested and works.

Most of the code for the entire project has been written.

What I was going to do next (in this order) was:
   
    1.  Test:
    	a.  That the data transferred accurately.
	b.  That the organization of the data being transferred is as I
	understand it.
    2.  Add and test transferring > 1 page of data.  If 3b is AIUI, this is
       trivial.
    3.  Restore some of the configuration dependencies in the makefiles in qemu,
        and set them in the configure file.  (See notes below)
    4.  Fix the white space in the different files.  

II.  APPROACH

I re-used a fair amount of virtio 9p code to produce the above.
In this respect, the front-end required more code, but less modification to
existing code.

II. A.  Front-end
When I began, I assumed the code should eventually reside in net/9p.
Only one file in this directory trans_virtio.c contains code that is virtio
specific.  The files client.c and mod.c, while technically part of virtio,
could be used as is, and my code calls functions in these files.

What I realized, in writing the backend, is that virtio provided the code that
takes the initial client request, massages it, and "unmarshals" these requests
in the backend.  Client.c and mod.c, do this massaging in the front-end.  The
format of the data that client.c sends in a request is easily moved to a
scatter-gather list.

p9_front_driver.c contains all the functions specified in the xenbus_driver
struct.  The module initialization registers the front end with xenbus, and
calls the 9p initialization code (in trans_xen9p.c) to register the 9p
transport.

The functions in trans_xen9p.c mimic their counterparts in trans_virtio.c, in
that they provide the same functionality.  They are the interface between the
functions in client.c and the xen protocol.  They receive requests from the client
and call functions in p9_front.c to issue the request.  The function in
trans_xen9p.c, that handles these requests, is specified in the 9p initialization
code.

p9_front.c contains the code that talks to the back end.  It contains the
p9_connect function, which is invoked when the front and back-end are connected.  It
also manages the ring, and the grant table (which, at present has only one entry),
receives requests from trans_xen9p.c, transfers the data into a shared page,
and issues the request to the backend.  When it receives a response, it passes
the response information to req_done (in trans_xen9p.c), which processes it, and calls the client callback function.

FYI - in linux/fs/fsdev there is the core of the 9p client code.  I did not touch
this code.

II. B. Backend
This was more complicated so I'll break it down into Background and What I did.

Background:
The virtio 9p backend code is composed of many files:  virtio-9p.c contains most
of the code to interpret the request from the client, and a few virtio-specific
functions.  virtio-9p-xattr.c and virtio-9p-xattr-user.c handle extended
attribute requests.  These files reside in qemu/hw/9pfs.

virtio-9p-co*.c contain co-routines for all the functions in virtio-9p.c and
virtio-xattr.c.

virtio-9p-proxy.c, virtio-9p-synth.c, virtio-9p-local.c implement the various
file systems that virtio supports.

virtio-9p-device.c contains an implementation of the qemu object model for 9p.

What I Did:

I broke virtio-9p.c into two files:  virtio-9p.c and v9fs_server.c.
virtio-9p.c now contains three functions:  complete_pdu, handle_9p_output, and a
"constructor", virtio_9p_set_fd_limit.

I created a "wrapper"header file for virtio-9p.h, v9fs_server.h.  This
wrapper creates a container for the V9fsState struct, V9GenericFsState,
that contains a pointer to xen-specific data, and a pointer to a complete
function, that can replace complete_pdu.  There are other function pointers that
I commented out, because they turned out not to be necessary.

v9pfs_server.h also defines GENERIC_9P_SERVER.  In virtio-9p.h, if
GENERIC_9P_SERVER is defined, complete_pdu is defined to translate
to the complete function in V9GenericFsState; if GENERIC_9P_SERVER  is not
defined, complete_pdu is simply declared.  This is the only ifdef I added.

I wrote a new file xen-9p.c that contains a complete_pdu function,
xen_complete_pdu, xen_pdu_handle, and set_fd_limit.  pdu's are the data
structures that are passed among all the functions that handle file requests.
They contain the meta-data necessary to process, and keep track of the request.

xen_pdu_handle takes a requests and creates a pdu for the existing functions to
handle the request.

xen_complete_pdu takes a completed request, frees memory and calls
p9_send_response in xen_9pif.c to handle the response.

The rest of xen_9pif.c contains the code that interacts directly with the xenbus
and indirectly with the front-end.  Much of it was modeled after xen_disk.c, and
tested when I was just passing small amounts of data back and forth.

The backend also contains files in qemu/fsdev that set up the QemuOptsList
tables and massage the data.  I added qemu-xen-fsdev-opts.c, that was patterned
after qemu-fsdev-opts.c.  I also modified qemu-fsdev.c to add debugging
statements.

I modified vl.c and qemu-options.def, in the qemu directory to match what
was done for virtio.  Some of this works, but not all of it.

I also modified xen_backend.h and xen_pv_machine.c to recognize this new
backend.  This code works.

I mentioned I modified the Makefiles, in qemu/hw/9pfs, qemu/fsdev, in qemu/hw,
and in qemu, inappropriately.  I did not learn until the end of my internship
exactly how to manipulate the configure files.  Therefore, I removed or changed
configuration dependencies, rather than change the makefiles.  This should be
fixed.

Some FIXME statements that I put in to remind myself what I needed to do,
actually got done, but I didn't remove the comment.

III.  How to Build My Project

My code now resides only on my github:  lindajac.
The front-end code is in lindajac/lindap9/p9/p9front.

To build it, assemble the front end files (p9_front.c p9_front_driver.c
trans_xen9p.c and p9.h) in a directory with the Makefile.  Copy from
linux/net/9p, the header file trans_common.h, to this directory.
This obviously has to change, when the front end has a permanent home directory.

Type make.  This will give you the front-end module.

My backend resides under the github: lindajac/qemu on the branch mychanges.

Because so much time has passed since my internship, it would have been more
work than I was willing to do for free, for me to sync my project with the
current qemu.   Therefore, this needs to be done.  The files that need to be
merged manually are qemu/vl.c and qemu/hw/9pfs/virtio-9p.c.  Given this caveat,
if you checkout this branch, it will build, and create a usable qemu.

IV.  How to 'Use' My Project

Here are the steps I follow.  These steps have to be followed to test or to run
the Xen 9p system.

1.  Create and pause a guest ( -p option ).
2.  Run xenstore-ls and edit the file to see what the domain number of the guest
is.
3.  Change the xenstore entries per the statements in the shell script
lindap9/p9/p9front/p9xsupdate.sh.  The parameter to this script is the
domain number of the guest.
4.  Start qemu.  The qemu line I have been using (in a shell script) is
<your path to qemu-system-i386> -xen-domid $1 -chardev socket,\
id=libxl-cmd,path=/var/run/xen/qmp-libxl-$1,server,nowait -no-shutdown\
-mon chardev=libxl-cmd,mode=control -chardev socket,id=libxenstat-cmd,\
path=/var/run/xen/qmp-libxenstat-$1,server,nowait -mon chardev=libxenstat-cmd,\
mode=control -nodefaults -xen-attach -name testg -vnc 127.0.0.1:0,to=99
-display none -machine xenpv -m 2048\
-xen9pfs local,security_model=passthrough,id=fsdev0,path=/mnt,mount_tag=P9Mount

where the last line is specific to 9p; all the rest is copied from the qemu
parameters I captured using a shell script, early in this project.  $1 is the
guest domain.
5.  Unpause the guest.
6.  xl console <guest>
7.  enter password.
8.  scp the .ko of the front end to the guest
9.  To run the front-end you need to modprobe my .ko, as well as the kernel modules
linux/net/9p and linux/fs/9p, but they do not need to be built specially, for
things to work.

That's it.




[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel