[lustre-devel] A new drivers/staging/lustre

[lustre-devel] A new drivers/staging/lustre

[NeilBrown]

Greg's patch to remove lustre has now landed in this staging-next tree,
so I suspect it will get to Linus before too long.  So I have to find a
new place to work on lustre.

I've added 2 branches to
   git://git.neil.brown.name/linux

lustre:
   is based on Greg's patch that removes lustre, and starts with a
   revert of the patch, followed by a merge of v4.17.
   I plan to merge each release and RC from Linus, and also
   add lustre patches that I think are "ready".  That will usually mean
   they have been posted to this list at least a week earlier, and
   have not had a credible negative response (Acks and Reviewed-by
   would be nice).
   I plan to update this branch about once a week, and to never rebase
   it.

lustre-testing:
   is based on 'lustre' and has most of my current lustre-related work.
   It includes assorted patches that are not specifically for lustre
   (rhashtables mostly at the moment).  Patches will move from here
   to 'lustre' or to mainline when they are ready.
   I plan to update this branch on most days that I work on Lustre,
   and expect it to rebase frequently.

I'm happy to review and, if acceptable, apply patches from other
developers.  I have fairly high standards, but if I don't accept your
patch I'll explain why and possible help fix it.
I'm happy to accept enhancements and new features, but these need
to be of a quality that would be accepted upstream.
I'm only interested in client-side code at present - nothing that is
only used on the server.  I do want to include server-side eventually,
but I need some focus for now.

I hope to get to a stage where the code is of suitable quality that I
can submit it to Linux as a new filesystem.  I hope that will happen
this year.

I hope we can continue to work together to make all this happen.
(That's enough hope for now, time to get back to code).

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180607/cee8d602/attachment.sig>

[lustre-devel] A new drivers/staging/lustre

[James Simmons]

> Greg's patch to remove lustre has now landed in this staging-next tree,
> so I suspect it will get to Linus before too long.  So I have to find a
> new place to work on lustre.
> 
> I've added 2 branches to
>    git://git.neil.brown.name/linux
> 
> lustre:
>    is based on Greg's patch that removes lustre, and starts with a
>    revert of the patch, followed by a merge of v4.17.
>    I plan to merge each release and RC from Linus, and also
>    add lustre patches that I think are "ready".  That will usually mean
>    they have been posted to this list at least a week earlier, and
>    have not had a credible negative response (Acks and Reviewed-by
>    would be nice).
>    I plan to update this branch about once a week, and to never rebase
>    it.

I know Oleg also started to play with a tree but I don't know if he can
keep it up like you can. I added the parties interested so they can bless
this tree if they want. Mainly Oleg wanted to see what breaks when moving
to fs directory and the proper UAPI headers directory as well.

Please be patience with me. I normally do this work on the weekend. I put
it into my test bed and try it out. Getting reviews can at times be 
challenging to get. Hopefully people at Cray and Intel are willing to
step in for this as well. Patrick from Cray has been most helpful.

> lustre-testing:
>    is based on 'lustre' and has most of my current lustre-related work.
>    It includes assorted patches that are not specifically for lustre
>    (rhashtables mostly at the moment).  Patches will move from here
>    to 'lustre' or to mainline when they are ready.
>    I plan to update this branch on most days that I work on Lustre,
>    and expect it to rebase frequently.

I had question about that. Some things in Lustre could in theory be merged
into the linux kernel proper. Can that be done still?
 
> I'm happy to review and, if acceptable, apply patches from other
> developers.  I have fairly high standards, but if I don't accept your
> patch I'll explain why and possible help fix it.

Also long as you talk to me :-) I'm an easy person to work with. If I 
refuse a patch with do the same. I might sometimes seem irrational 
but I have valid reasons. Well at least in my head.

We need to really layout the roadmap.

> I'm happy to accept enhancements and new features, but these need
> to be of a quality that would be accepted upstream.

Absolutely. This should be music to some peoples ears. 

> I'm only interested in client-side code at present - nothing that is
> only used on the server.  I do want to include server-side eventually,
> but I need some focus for now.

Make sense. Anyways the backend file systems used are ldiskfs which is
a heavily modified ext4 filesystem and ZFS on the server side. I doubt
the kernel would accept ZFS backend suppport and the changes Lustre
does to ext4 have been mostly merged but a few pieces are missing.
So pushing the server code at this point wouldn't benefit us.

> I hope to get to a stage where the code is of suitable quality that I
> can submit it to Linux as a new filesystem.  I hope that will happen
> this year.

Are you thinking going back into staging or straight to the fs tree

> I hope we can continue to work together to make all this happen.
> (That's enough hope for now, time to get back to code).

[lustre-devel] A new drivers/staging/lustre

[NeilBrown]

On Thu, Jun 07 2018, James Simmons wrote:

>> Greg's patch to remove lustre has now landed in this staging-next tree,
>> so I suspect it will get to Linus before too long.  So I have to find a
>> new place to work on lustre.
>> 
>> I've added 2 branches to
>>    git://git.neil.brown.name/linux
>> 
>> lustre:
>>    is based on Greg's patch that removes lustre, and starts with a
>>    revert of the patch, followed by a merge of v4.17.
>>    I plan to merge each release and RC from Linus, and also
>>    add lustre patches that I think are "ready".  That will usually mean
>>    they have been posted to this list at least a week earlier, and
>>    have not had a credible negative response (Acks and Reviewed-by
>>    would be nice).
>>    I plan to update this branch about once a week, and to never rebase
>>    it.
>
> I know Oleg also started to play with a tree but I don't know if he can
> keep it up like you can. I added the parties interested so they can bless
> this tree if they want. Mainly Oleg wanted to see what breaks when moving
> to fs directory and the proper UAPI headers directory as well.

I'm certainly happy to contribute to a different tree instead, if
someone else is working towards getting lustre code suitable for
upstream.  I created a tree myself because I find that saying "we should"
is not nearly as effective as saying "I have".

>
> Please be patience with me. I normally do this work on the weekend. I put
> it into my test bed and try it out. Getting reviews can at times be 
> challenging to get. Hopefully people at Cray and Intel are willing to
> step in for this as well. Patrick from Cray has been most helpful.
>
>> lustre-testing:
>>    is based on 'lustre' and has most of my current lustre-related work.
>>    It includes assorted patches that are not specifically for lustre
>>    (rhashtables mostly at the moment).  Patches will move from here
>>    to 'lustre' or to mainline when they are ready.
>>    I plan to update this branch on most days that I work on Lustre,
>>    and expect it to rebase frequently.
>
> I had question about that. Some things in Lustre could in theory be merged
> into the linux kernel proper. Can that be done still?

What things?

If it measurably benefits the kernel proper, then I suspect it might be
worth submitting.  Things can go direct without going though staging -
they just have to be of good quality with good justification (and
sometimes lots of patience).

>  
>> I'm happy to review and, if acceptable, apply patches from other
>> developers.  I have fairly high standards, but if I don't accept your
>> patch I'll explain why and possible help fix it.
>
> Also long as you talk to me :-) I'm an easy person to work with. If I 
> refuse a patch with do the same. I might sometimes seem irrational 
> but I have valid reasons. Well at least in my head.
>
> We need to really layout the roadmap.

I have very little faith in road maps - I prefer to make steps.  Once we
have made all the steps, we can look back and see what the map looked
like in retrospect.

The most I'm interested in is "client first, then server".
But feel free to propose something - it is helps you then it could be
useful.

>
>> I'm happy to accept enhancements and new features, but these need
>> to be of a quality that would be accepted upstream.
>
> Absolutely. This should be music to some peoples ears. 
>
>> I'm only interested in client-side code at present - nothing that is
>> only used on the server.  I do want to include server-side eventually,
>> but I need some focus for now.
>
> Make sense. Anyways the backend file systems used are ldiskfs which is
> a heavily modified ext4 filesystem and ZFS on the server side. I doubt
> the kernel would accept ZFS backend suppport and the changes Lustre
> does to ext4 have been mostly merged but a few pieces are missing.
> So pushing the server code at this point wouldn't benefit us.
>
>> I hope to get to a stage where the code is of suitable quality that I
>> can submit it to Linux as a new filesystem.  I hope that will happen
>> this year.
>
> Are you thinking going back into staging or straight to the fs tree

Greg has said he doesn't want it in staging.  So no, I'm not thinking of
anything going to staging.  I'm thinking of getting enough of a client
in reasonable shape that people can review it without feeling sick or
getting angry.

Thanks,
NeilBrown


>
>> I hope we can continue to work together to make all this happen.
>> (That's enough hope for now, time to get back to code).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180607/b3f1a5b3/attachment.sig>

[lustre-devel] A new drivers/staging/lustre

[Doug Oucharek]

What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:


  *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
  *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).

For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.

There are some very big (as in code size) features missing from upstream.  For example, Multi-Rail.  When should that be pushed relative to code cleanups?

Doug

On Jun 7, 2018, at 4:58 AM, NeilBrown <neilb at suse.com<mailto:neilb@suse.com>> wrote:

On Thu, Jun 07 2018, James Simmons wrote:

Greg's patch to remove lustre has now landed in this staging-next tree,
so I suspect it will get to Linus before too long.  So I have to find a
new place to work on lustre.

I've added 2 branches to
  git://git.neil.brown.name/linux

lustre:
  is based on Greg's patch that removes lustre, and starts with a
  revert of the patch, followed by a merge of v4.17.
  I plan to merge each release and RC from Linus, and also
  add lustre patches that I think are "ready".  That will usually mean
  they have been posted to this list at least a week earlier, and
  have not had a credible negative response (Acks and Reviewed-by
  would be nice).
  I plan to update this branch about once a week, and to never rebase
  it.

I know Oleg also started to play with a tree but I don't know if he can
keep it up like you can. I added the parties interested so they can bless
this tree if they want. Mainly Oleg wanted to see what breaks when moving
to fs directory and the proper UAPI headers directory as well.

I'm certainly happy to contribute to a different tree instead, if
someone else is working towards getting lustre code suitable for
upstream.  I created a tree myself because I find that saying "we should"
is not nearly as effective as saying "I have".


Please be patience with me. I normally do this work on the weekend. I put
it into my test bed and try it out. Getting reviews can at times be
challenging to get. Hopefully people at Cray and Intel are willing to
step in for this as well. Patrick from Cray has been most helpful.

lustre-testing:
  is based on 'lustre' and has most of my current lustre-related work.
  It includes assorted patches that are not specifically for lustre
  (rhashtables mostly at the moment).  Patches will move from here
  to 'lustre' or to mainline when they are ready.
  I plan to update this branch on most days that I work on Lustre,
  and expect it to rebase frequently.

I had question about that. Some things in Lustre could in theory be merged
into the linux kernel proper. Can that be done still?

What things?

If it measurably benefits the kernel proper, then I suspect it might be
worth submitting.  Things can go direct without going though staging -
they just have to be of good quality with good justification (and
sometimes lots of patience).


I'm happy to review and, if acceptable, apply patches from other
developers.  I have fairly high standards, but if I don't accept your
patch I'll explain why and possible help fix it.

Also long as you talk to me :-) I'm an easy person to work with. If I
refuse a patch with do the same. I might sometimes seem irrational
but I have valid reasons. Well at least in my head.

We need to really layout the roadmap.

I have very little faith in road maps - I prefer to make steps.  Once we
have made all the steps, we can look back and see what the map looked
like in retrospect.

The most I'm interested in is "client first, then server".
But feel free to propose something - it is helps you then it could be
useful.


I'm happy to accept enhancements and new features, but these need
to be of a quality that would be accepted upstream.

Absolutely. This should be music to some peoples ears.

I'm only interested in client-side code at present - nothing that is
only used on the server.  I do want to include server-side eventually,
but I need some focus for now.

Make sense. Anyways the backend file systems used are ldiskfs which is
a heavily modified ext4 filesystem and ZFS on the server side. I doubt
the kernel would accept ZFS backend suppport and the changes Lustre
does to ext4 have been mostly merged but a few pieces are missing.
So pushing the server code at this point wouldn't benefit us.

I hope to get to a stage where the code is of suitable quality that I
can submit it to Linux as a new filesystem.  I hope that will happen
this year.

Are you thinking going back into staging or straight to the fs tree

Greg has said he doesn't want it in staging.  So no, I'm not thinking of
anything going to staging.  I'm thinking of getting enough of a client
in reasonable shape that people can review it without feeling sick or
getting angry.

Thanks,
NeilBrown



I hope we can continue to work together to make all this happen.
(That's enough hope for now, time to get back to code).
_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180607/dee00f7d/attachment-0001.html>

[lustre-devel] A new drivers/staging/lustre

[Patrick Farrell]

Doug,

Another thought about the core reason.

Commitment to this.  The existing code state (weird, abandoned 2.4.something code) and merge rules seemingly made it impossible to shift development in this direction in any meaningful way, so perhaps it's chicken and egg...  but as long as Lustre is released and developed primarily out of tree, I can't see this working.  Would it just be a "sync everything but still do releases" approach?  Is that viable?  Etc.

Thoughts appreciated.

- Patrick


________________________________
From: lustre-devel <lustre-devel-bounces@lists.lustre.org> on behalf of Doug Oucharek <doucharek@cray.com>
Sent: Thursday, June 7, 2018 8:07:44 AM
To: NeilBrown
Cc: Oleg Drokin; lustre-devel
Subject: Re: [lustre-devel] A new drivers/staging/lustre

What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:


  *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
  *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).

For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.

There are some very big (as in code size) features missing from upstream.  For example, Multi-Rail.  When should that be pushed relative to code cleanups?

Doug

On Jun 7, 2018, at 4:58 AM, NeilBrown <neilb at suse.com<mailto:neilb@suse.com>> wrote:

On Thu, Jun 07 2018, James Simmons wrote:

Greg's patch to remove lustre has now landed in this staging-next tree,
so I suspect it will get to Linus before too long.  So I have to find a
new place to work on lustre.

I've added 2 branches to
  git://git.neil.brown.name/linux

lustre:
  is based on Greg's patch that removes lustre, and starts with a
  revert of the patch, followed by a merge of v4.17.
  I plan to merge each release and RC from Linus, and also
  add lustre patches that I think are "ready".  That will usually mean
  they have been posted to this list at least a week earlier, and
  have not had a credible negative response (Acks and Reviewed-by
  would be nice).
  I plan to update this branch about once a week, and to never rebase
  it.

I know Oleg also started to play with a tree but I don't know if he can
keep it up like you can. I added the parties interested so they can bless
this tree if they want. Mainly Oleg wanted to see what breaks when moving
to fs directory and the proper UAPI headers directory as well.

I'm certainly happy to contribute to a different tree instead, if
someone else is working towards getting lustre code suitable for
upstream.  I created a tree myself because I find that saying "we should"
is not nearly as effective as saying "I have".


Please be patience with me. I normally do this work on the weekend. I put
it into my test bed and try it out. Getting reviews can at times be
challenging to get. Hopefully people at Cray and Intel are willing to
step in for this as well. Patrick from Cray has been most helpful.

lustre-testing:
  is based on 'lustre' and has most of my current lustre-related work.
  It includes assorted patches that are not specifically for lustre
  (rhashtables mostly at the moment).  Patches will move from here
  to 'lustre' or to mainline when they are ready.
  I plan to update this branch on most days that I work on Lustre,
  and expect it to rebase frequently.

I had question about that. Some things in Lustre could in theory be merged
into the linux kernel proper. Can that be done still?

What things?

If it measurably benefits the kernel proper, then I suspect it might be
worth submitting.  Things can go direct without going though staging -
they just have to be of good quality with good justification (and
sometimes lots of patience).


I'm happy to review and, if acceptable, apply patches from other
developers.  I have fairly high standards, but if I don't accept your
patch I'll explain why and possible help fix it.

Also long as you talk to me :-) I'm an easy person to work with. If I
refuse a patch with do the same. I might sometimes seem irrational
but I have valid reasons. Well at least in my head.

We need to really layout the roadmap.

I have very little faith in road maps - I prefer to make steps.  Once we
have made all the steps, we can look back and see what the map looked
like in retrospect.

The most I'm interested in is "client first, then server".
But feel free to propose something - it is helps you then it could be
useful.


I'm happy to accept enhancements and new features, but these need
to be of a quality that would be accepted upstream.

Absolutely. This should be music to some peoples ears.

I'm only interested in client-side code at present - nothing that is
only used on the server.  I do want to include server-side eventually,
but I need some focus for now.

Make sense. Anyways the backend file systems used are ldiskfs which is
a heavily modified ext4 filesystem and ZFS on the server side. I doubt
the kernel would accept ZFS backend suppport and the changes Lustre
does to ext4 have been mostly merged but a few pieces are missing.
So pushing the server code at this point wouldn't benefit us.

I hope to get to a stage where the code is of suitable quality that I
can submit it to Linux as a new filesystem.  I hope that will happen
this year.

Are you thinking going back into staging or straight to the fs tree

Greg has said he doesn't want it in staging.  So no, I'm not thinking of
anything going to staging.  I'm thinking of getting enough of a client
in reasonable shape that people can review it without feeling sick or
getting angry.

Thanks,
NeilBrown



I hope we can continue to work together to make all this happen.
(That's enough hope for now, time to get back to code).
_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org<mailto:lustre-devel@lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180607/8f0662bb/attachment-0001.html>

[lustre-devel] A new drivers/staging/lustre

[NeilBrown]

On Thu, Jun 07 2018, Doug Oucharek wrote:

> What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:
>
>
>   *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
>   *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).
>
> For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.

My (undoubtedly biased) perspective on the history of lustre in staging
goes like this:
There are two things needed for some out-of-tree code to get into
mainline Linux:  the code needs to be integrated and the community needs
to be integrated (or a new sub-community needs to form).
In the case of lustre, the code was never really integrated because the
community never really tried to integrate.  Integrating and becoming
part of the Linux community takes time and effort, and it is quite
possible that management for various developers didn't allocate enough
time over a long enough period.  Integrating also requires a change in
attitude and I don't see much evidence of that.  I see clear evidence of
an "us and them" attitude among (some) lustre developers - almost as
though upstream linux is hostile territory full of unfriendly developers
who always reject our excellent code (even though they have lots of
horrible code themselves).  *We* need to see ourselves as part of the
Linux community, and we need to care about all of Linux as though it was
all ours (it *is* all ours, but *we* are a much larger group now).

Yes, the current code needs to be improved, bugs need to be fixed, and
features need to be added.  The order in which these is done is not the
most important things - if it were, Greg would have never accepted any
new features.  However he *did* accept them, but tried to remind the
lustre developers that there was other work to do.

Working together in one (single) community requires give-and-take.
Greg's behaviour as just described seems to be evidence of
give-and-take.  I think he kicked lustre out of staging because he
concluded that he was never going to get the matching give-and-take in
return.

So to answer your opening question, my focus for this tree is to train
any lustre developers who wish to engage about how to be part of the
Linux community.  As I've already said - I will accept features but I
prefer cleanups first.  I don't want to try to explain further than that
because it will be too hypothetical and unhelpful.  We - the Linux
community - don't work in hypotheticals.  We work with concrete objects
like patches.  So send me a patch and I will tell you what I think of
that specific patch.  It is up to you to generalise what I say to other
patches.  It might also be up to you to argue your case and tell me why
I'm wrong.  I'll be patient (because good upstream maintainers are) but
patience doesn't last forever (for Greg, it lasted about 5 years - I
hope mine won't be tried to that extent).

>
> There are some very big (as in code size) features missing from
> upstream.  For example, Multi-Rail.  When should that be pushed
> relative to code cleanups?

Never add features to ugly code - fix the code first.
The doesn't mean you cannot add any feature to lustre until all of
lustre is beautify.  But it does mean that if I can see in a patch some
ugly code and a new feature, then I won't be happy.  First clean up just
enough of the ugliness so that it won't be visible in the patch that
adds the feature.
But again, this is getting a bit too hypothetical.   If you care about a
feature, then post a patch.  We can take it from there.  The fact that
you care enough to post a patch cares significant weight - a lot more
weight than just asking about some feature.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180608/61bd6254/attachment.sig>

[lustre-devel] A new drivers/staging/lustre

[NeilBrown]

On Thu, Jun 07 2018, Patrick Farrell wrote:

> Doug,
>
> Another thought about the core reason.
>
> Commitment to this.  The existing code state (weird, abandoned 2.4.something code) and merge rules seemingly made it impossible to shift development in this direction in any meaningful way, so perhaps it's chicken and egg...  but as long as Lustre is released and developed primarily out of tree, I can't see this working.  Would it just be a "sync everything but still do releases" approach?  Is that viable?  Etc.
>
> Thoughts appreciated.

Yes, commitment is important - from management was well as developers.
If you think getting lustre upstream is important then make sure you
manager understands why.

Having development in two separate tree  - one heading for upstream
inclusion, the other used by customers - could be a problem and
certainly needs an end date, but I think it is our best option at the
moment (even though Greg claimed it was part of his reason for ejecting
lustre from staging).

I think that imposing an upstream development module on (what I've
decided to call) legacy-lustre would be a constant battle with a low
success rate.  Conversely moving code from legacy-lustre into an
upstream work-flow should be quite manageable providing time and skills
are available.

I would really like to be able to point to a location in the
legacy-lustre git history and be able to say "linux-lustre has all
features up to this point", and then to progressively copy features
across and move that point forward.  As a feature is copied, we make
sure it conforms with upstream standards.  I won't be doing any of this
until I'm really happy with what is already in linux-lustre, but I won't
try to set priorities for other people.

Once we get to the point where linux-lustre has feature-parity with
legacy-lustre, then we can discuss doing development in linux-lustre
first, and copying it across to legacy-lustre only as long as people
care.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180608/7da00f5f/attachment.sig>

[lustre-devel] A new drivers/staging/lustre

[Dilger, Andreas]

On Jun 6, 2018, at 20:29, NeilBrown <neilb@suse.com> wrote:
> Greg's patch to remove lustre has now landed in this staging-next tree,
> so I suspect it will get to Linus before too long.  So I have to find a
> new place to work on lustre.
> 
> I've added 2 branches to
>   git://git.neil.brown.name/linux
> 
> lustre:
>   is based on Greg's patch that removes lustre, and starts with a
>   revert of the patch, followed by a merge of v4.17.
>   I plan to merge each release and RC from Linus, and also
>   add lustre patches that I think are "ready".  That will usually mean
>   they have been posted to this list at least a week earlier, and
>   have not had a credible negative response (Acks and Reviewed-by
>   would be nice).
>   I plan to update this branch about once a week, and to never rebase
>   it.
> 
> lustre-testing:
>   is based on 'lustre' and has most of my current lustre-related work.
>   It includes assorted patches that are not specifically for lustre
>   (rhashtables mostly at the moment).  Patches will move from here
>   to 'lustre' or to mainline when they are ready.
>   I plan to update this branch on most days that I work on Lustre,
>   and expect it to rebase frequently.

We also have a clone of Greg's staging branch on our Gerrit server,
which could be replaced with a "vanilla" upstream kernel clone that
holds a copy of the for-upstream Lustre tree.

One benefit of hosting the tree on the Gerrit server is that it is
much easier to get automated testing of the patches (a lot or a little)
to ensure that the code doesn't break anything obvious.

> I'm happy to review and, if acceptable, apply patches from other
> developers.  I have fairly high standards, but if I don't accept your
> patch I'll explain why and possible help fix it.
> I'm happy to accept enhancements and new features, but these need
> to be of a quality that would be accepted upstream.
> I'm only interested in client-side code at present - nothing that is
> only used on the server.  I do want to include server-side eventually,
> but I need some focus for now.
> 
> I hope to get to a stage where the code is of suitable quality that I
> can submit it to Linux as a new filesystem.  I hope that will happen
> this year.

I think one major drawback of starting with the from-staging version of
the code is that this is significantly behind the out-of-tree master
code and would need a lot of effort to catch up.

I think the only way we are going to make this work is if the cleanups
go into the same repo as the ongoing development.  Otherwise, we'll be
back in the same situation as we are now where there are two different
versions of the code and we need to move patches back and forth between
them.

On the plus side, this will avoid code drift, and the extra efforts of
porting patches back and forth.  On the minus side, it means (on some
fronts) that the code will regress from what is upstream, because we
don't have all of the trivial whitespace cleanups and other drive-by
kernel newbie patches that went into the upstream client.  However, it
does mean that those cleanups would now become part of the single code
tree and not be lost again in the future.

This would also mean some restructuring of the current Lustre tree so
that it could build as a drop-in at linux/fs/lustre but that is probably
a good idea for the long term in any case.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation

[lustre-devel] A new drivers/staging/lustre

[Doug Oucharek]

Does this mean we would be pushing patches to Gerrit rather than having to email them out?  I believe for the test system to work automatically, the answer is yes.

Doug

________________________________
From: lustre-devel <lustre-devel-bounces@lists.lustre.org> on behalf of Dilger, Andreas <andreas.dilger@intel.com>
Sent: Thursday, June 7, 2018 5:46:26 PM
To: NeilBrown
Cc: lustre-devel
Subject: Re: [lustre-devel] A new drivers/staging/lustre

On Jun 6, 2018, at 20:29, NeilBrown <neilb@suse.com> wrote:
> Greg's patch to remove lustre has now landed in this staging-next tree,
> so I suspect it will get to Linus before too long.  So I have to find a
> new place to work on lustre.
>
> I've added 2 branches to
>   git://git.neil.brown.name/linux
>
> lustre:
>   is based on Greg's patch that removes lustre, and starts with a
>   revert of the patch, followed by a merge of v4.17.
>   I plan to merge each release and RC from Linus, and also
>   add lustre patches that I think are "ready".  That will usually mean
>   they have been posted to this list at least a week earlier, and
>   have not had a credible negative response (Acks and Reviewed-by
>   would be nice).
>   I plan to update this branch about once a week, and to never rebase
>   it.
>
> lustre-testing:
>   is based on 'lustre' and has most of my current lustre-related work.
>   It includes assorted patches that are not specifically for lustre
>   (rhashtables mostly at the moment).  Patches will move from here
>   to 'lustre' or to mainline when they are ready.
>   I plan to update this branch on most days that I work on Lustre,
>   and expect it to rebase frequently.

We also have a clone of Greg's staging branch on our Gerrit server,
which could be replaced with a "vanilla" upstream kernel clone that
holds a copy of the for-upstream Lustre tree.

One benefit of hosting the tree on the Gerrit server is that it is
much easier to get automated testing of the patches (a lot or a little)
to ensure that the code doesn't break anything obvious.

> I'm happy to review and, if acceptable, apply patches from other
> developers.  I have fairly high standards, but if I don't accept your
> patch I'll explain why and possible help fix it.
> I'm happy to accept enhancements and new features, but these need
> to be of a quality that would be accepted upstream.
> I'm only interested in client-side code at present - nothing that is
> only used on the server.  I do want to include server-side eventually,
> but I need some focus for now.
>
> I hope to get to a stage where the code is of suitable quality that I
> can submit it to Linux as a new filesystem.  I hope that will happen
> this year.

I think one major drawback of starting with the from-staging version of
the code is that this is significantly behind the out-of-tree master
code and would need a lot of effort to catch up.

I think the only way we are going to make this work is if the cleanups
go into the same repo as the ongoing development.  Otherwise, we'll be
back in the same situation as we are now where there are two different
versions of the code and we need to move patches back and forth between
them.

On the plus side, this will avoid code drift, and the extra efforts of
porting patches back and forth.  On the minus side, it means (on some
fronts) that the code will regress from what is upstream, because we
don't have all of the trivial whitespace cleanups and other drive-by
kernel newbie patches that went into the upstream client.  However, it
does mean that those cleanups would now become part of the single code
tree and not be lost again in the future.

This would also mean some restructuring of the current Lustre tree so
that it could build as a drop-in at linux/fs/lustre but that is probably
a good idea for the long term in any case.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation







_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180607/e8ac9ba2/attachment-0001.html>

[lustre-devel] A new drivers/staging/lustre

[Dilger, Andreas]

On Jun 7, 2018, at 15:38, NeilBrown <neilb@suse.com> wrote:
> 
> On Thu, Jun 07 2018, Doug Oucharek wrote:
> 
>> What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:
>> 
>> 
>>  *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
>>  *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).
>> 
>> For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.
> 
> My (undoubtedly biased) perspective on the history of lustre in staging
> goes like this:
> There are two things needed for some out-of-tree code to get into
> mainline Linux:  the code needs to be integrated and the community needs
> to be integrated (or a new sub-community needs to form).
> In the case of lustre, the code was never really integrated because the
> community never really tried to integrate.

One of the issues here was that the group (not Intel) that submitted the
Lustre code to the staging tree promptly abandoned it for a couple of
years after they submitted it upstream, after promising the community
that they were in it for the long run.  That put the upstream integration
behind the eight-ball from the start.

>  Integrating and becoming
> part of the Linux community takes time and effort, and it is quite
> possible that management for various developers didn't allocate enough
> time over a long enough period.  Integrating also requires a change in
> attitude and I don't see much evidence of that.  I see clear evidence of
> an "us and them" attitude among (some) lustre developers - almost as
> though upstream linux is hostile territory full of unfriendly developers

Ah, but it *is* hostile territory, if you are not among the "in crowd".
Christoph can get any change he wants to be accepted, but if someone else
tries to push something similar it can be rejected outright or ignored
for months or years.

> who always reject our excellent code (even though they have lots of
> horrible code themselves).  *We* need to see ourselves as part of the
> Linux community, and we need to care about all of Linux as though it was
> all ours (it *is* all ours, but *we* are a much larger group now).
> 
> Yes, the current code needs to be improved, bugs need to be fixed, and
> features need to be added.  The order in which these is done is not the
> most important things - if it were, Greg would have never accepted any
> new features.  However he *did* accept them, but tried to remind the
> lustre developers that there was other work to do.
> 
> Working together in one (single) community requires give-and-take.
> Greg's behaviour as just described seems to be evidence of
> give-and-take.  I think he kicked lustre out of staging because he
> concluded that he was never going to get the matching give-and-take in
> return.
> 
> So to answer your opening question, my focus for this tree is to train
> any lustre developers who wish to engage about how to be part of the
> Linux community.  As I've already said - I will accept features but I
> prefer cleanups first.  I don't want to try to explain further than that
> because it will be too hypothetical and unhelpful.  We - the Linux
> community - don't work in hypotheticals.  We work with concrete objects
> like patches.  So send me a patch and I will tell you what I think of
> that specific patch.  It is up to you to generalise what I say to other
> patches.  It might also be up to you to argue your case and tell me why
> I'm wrong.  I'll be patient (because good upstream maintainers are) but
> patience doesn't last forever (for Greg, it lasted about 5 years - I
> hope mine won't be tried to that extent).

Like I said in my other email, I think having another fork of the Lustre
tree, especially one that is starting from two-year-old code is likely to
fail, because there will be twice as much effort spent to maintain the two
trees.  I'd rather see the cleanups and features go hand-in-hand into the
same tree.  I'd be thrilled to have more reviews done on the features before
they are landed, but we can't just stop all feature development for a year
or two (or five) while the code is merged into the upstream kernel.

>> There are some very big (as in code size) features missing from
>> upstream.  For example, Multi-Rail.  When should that be pushed
>> relative to code cleanups?
> 
> Never add features to ugly code - fix the code first.
> The doesn't mean you cannot add any feature to lustre until all of
> lustre is beautiful.  But it does mean that if I can see in a patch some
> ugly code and a new feature, then I won't be happy.  First clean up just
> enough of the ugliness so that it won't be visible in the patch that
> adds the feature.

The issue is that we _can't_ just stop the development of new code/features
for such a long time.  There are huge supercomputers being deployed or in
planning that depend on these new features, or they wouldn't have been
developed in the first place.

Consider if the NFSv4 spec was written and the code was developed, and you
were told you needed to go back to NFSv2 and start again?

> But again, this is getting a bit too hypothetical.   If you care about a
> feature, then post a patch.  We can take it from there.  The fact that
> you care enough to post a patch cares significant weight - a lot more
> weight than just asking about some feature.

We definitely aren't at the point of "asking for some feature to be developed".
At a minimum the starting point of the new upstream code needs to be the
current release, or any resources that could possibly be put towards improving
the Lustre code would be squandered on porting all of those patches to the
upstream tree.  I'm fine with spending time to improve the code that exists
today, but lets not start with a huge deficit from the outset.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation

[lustre-devel] A new drivers/staging/lustre

[Dilger, Andreas]

On Jun 7, 2018, at 16:53, Doug Oucharek <doucharek@cray.com> wrote:
> 
> Does this mean we would be pushing patches to Gerrit rather than having to email them out?  I believe for the test system to work automatically, the answer is yes.

I'm not an expert with Gerrit configuration, so I'm not sure if there is an
email-to-change gateway plugin for it or not.  I believe it can be configured
to send the full patch in emails.  I get an email for every patch submitted,
but I configure it to only deliver the commit message and not the whole patch.

Since everyone is using Git anyway, using "git push" to submit patches to Gerrit
for review (and forwards them to the list, if that is what people want) isn't
more effort than "git send-email" to send them to the list directly.

I think any branch which just takes patches without adequate testing in a
variety of configurations (e.g. DNE, ZFS/ldiskfs, TCP/IB, quota, recovery,
version interop, failover, etc.) is just going to accumulate small or large
breakages and will not be usable in production.

James can probably tell you how many times "simple" patches were landed into
the upstream kernel that ended up breaking something because they weren't
tested.  I don't expect everyone to be able to test all of those configurations
themselves, which is why it is good to have automated testing to do this for
every patch that is submitted.

I'd be *thrilled* if more organizations/developers provided automated testing
of patches (e.g. PPC, ARM, Cray networking, etc.), so it isn't like I'm trying
to monopolize the testing (far from it).  However, the reality is that there
*is* no other automated testing environment beyond what we have with the
existing Jenkins/Gerrit/autotest/Maloo infrastructure we have today.  It is
imperfect for sure, but a gigantic leap beyond not having anything at all.

Cheers, Andreas

> From: Dilger, Andreas <andreas.dilger@intel.com>
> Sent: Thursday, June 7, 2018 5:46:26 PM
> To: NeilBrown
> Cc: lustre-devel
> Subject: Re: [lustre-devel] A new drivers/staging/lustre
>  
> On Jun 6, 2018, at 20:29, NeilBrown <neilb@suse.com> wrote:
> > Greg's patch to remove lustre has now landed in this staging-next tree,
> > so I suspect it will get to Linus before too long.  So I have to find a
> > new place to work on lustre.
> > 
> > I've added 2 branches to
> >   git://git.neil.brown.name/linux
> > 
> > lustre:
> >   is based on Greg's patch that removes lustre, and starts with a
> >   revert of the patch, followed by a merge of v4.17.
> >   I plan to merge each release and RC from Linus, and also
> >   add lustre patches that I think are "ready".  That will usually mean
> >   they have been posted to this list at least a week earlier, and
> >   have not had a credible negative response (Acks and Reviewed-by
> >   would be nice).
> >   I plan to update this branch about once a week, and to never rebase
> >   it.
> > 
> > lustre-testing:
> >   is based on 'lustre' and has most of my current lustre-related work.
> >   It includes assorted patches that are not specifically for lustre
> >   (rhashtables mostly at the moment).  Patches will move from here
> >   to 'lustre' or to mainline when they are ready.
> >   I plan to update this branch on most days that I work on Lustre,
> >   and expect it to rebase frequently.
> 
> We also have a clone of Greg's staging branch on our Gerrit server,
> which could be replaced with a "vanilla" upstream kernel clone that
> holds a copy of the for-upstream Lustre tree.
> 
> One benefit of hosting the tree on the Gerrit server is that it is
> much easier to get automated testing of the patches (a lot or a little)
> to ensure that the code doesn't break anything obvious.
> 
> > I'm happy to review and, if acceptable, apply patches from other
> > developers.  I have fairly high standards, but if I don't accept your
> > patch I'll explain why and possible help fix it.
> > I'm happy to accept enhancements and new features, but these need
> > to be of a quality that would be accepted upstream.
> > I'm only interested in client-side code at present - nothing that is
> > only used on the server.  I do want to include server-side eventually,
> > but I need some focus for now.
> > 
> > I hope to get to a stage where the code is of suitable quality that I
> > can submit it to Linux as a new filesystem.  I hope that will happen
> > this year.
> 
> I think one major drawback of starting with the from-staging version of
> the code is that this is significantly behind the out-of-tree master
> code and would need a lot of effort to catch up.
> 
> I think the only way we are going to make this work is if the cleanups
> go into the same repo as the ongoing development.  Otherwise, we'll be
> back in the same situation as we are now where there are two different
> versions of the code and we need to move patches back and forth between
> them.
> 
> On the plus side, this will avoid code drift, and the extra efforts of
> porting patches back and forth.  On the minus side, it means (on some
> fronts) that the code will regress from what is upstream, because we
> don't have all of the trivial whitespace cleanups and other drive-by
> kernel newbie patches that went into the upstream client.  However, it
> does mean that those cleanups would now become part of the single code
> tree and not be lost again in the future.
> 
> This would also mean some restructuring of the current Lustre tree so
> that it could build as a drop-in at linux/fs/lustre but that is probably
> a good idea for the long term in any case.
> 
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Intel Corporation
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> lustre-devel mailing list
> lustre-devel at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation

[lustre-devel] A new drivers/staging/lustre

[Patrick Farrell]

For what it?s worth, I agree strongly with Andreas about getting this in sync with the current Intel branch FIRST.  There is far more additional code/history there in the form of already complete features and fixes than here in cleanups.

________________________________
From: lustre-devel <lustre-devel-bounces@lists.lustre.org> on behalf of Dilger, Andreas <andreas.dilger@intel.com>
Sent: Thursday, June 7, 2018 6:06:23 PM
To: NeilBrown
Cc: Drokin, Oleg; lustre-devel
Subject: Re: [lustre-devel] A new drivers/staging/lustre

On Jun 7, 2018, at 15:38, NeilBrown <neilb@suse.com> wrote:
>
> On Thu, Jun 07 2018, Doug Oucharek wrote:
>
>> What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:
>>
>>
>>  *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
>>  *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).
>>
>> For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.
>
> My (undoubtedly biased) perspective on the history of lustre in staging
> goes like this:
> There are two things needed for some out-of-tree code to get into
> mainline Linux:  the code needs to be integrated and the community needs
> to be integrated (or a new sub-community needs to form).
> In the case of lustre, the code was never really integrated because the
> community never really tried to integrate.

One of the issues here was that the group (not Intel) that submitted the
Lustre code to the staging tree promptly abandoned it for a couple of
years after they submitted it upstream, after promising the community
that they were in it for the long run.  That put the upstream integration
behind the eight-ball from the start.

>  Integrating and becoming
> part of the Linux community takes time and effort, and it is quite
> possible that management for various developers didn't allocate enough
> time over a long enough period.  Integrating also requires a change in
> attitude and I don't see much evidence of that.  I see clear evidence of
> an "us and them" attitude among (some) lustre developers - almost as
> though upstream linux is hostile territory full of unfriendly developers

Ah, but it *is* hostile territory, if you are not among the "in crowd".
Christoph can get any change he wants to be accepted, but if someone else
tries to push something similar it can be rejected outright or ignored
for months or years.

> who always reject our excellent code (even though they have lots of
> horrible code themselves).  *We* need to see ourselves as part of the
> Linux community, and we need to care about all of Linux as though it was
> all ours (it *is* all ours, but *we* are a much larger group now).
>
> Yes, the current code needs to be improved, bugs need to be fixed, and
> features need to be added.  The order in which these is done is not the
> most important things - if it were, Greg would have never accepted any
> new features.  However he *did* accept them, but tried to remind the
> lustre developers that there was other work to do.
>
> Working together in one (single) community requires give-and-take.
> Greg's behaviour as just described seems to be evidence of
> give-and-take.  I think he kicked lustre out of staging because he
> concluded that he was never going to get the matching give-and-take in
> return.
>
> So to answer your opening question, my focus for this tree is to train
> any lustre developers who wish to engage about how to be part of the
> Linux community.  As I've already said - I will accept features but I
> prefer cleanups first.  I don't want to try to explain further than that
> because it will be too hypothetical and unhelpful.  We - the Linux
> community - don't work in hypotheticals.  We work with concrete objects
> like patches.  So send me a patch and I will tell you what I think of
> that specific patch.  It is up to you to generalise what I say to other
> patches.  It might also be up to you to argue your case and tell me why
> I'm wrong.  I'll be patient (because good upstream maintainers are) but
> patience doesn't last forever (for Greg, it lasted about 5 years - I
> hope mine won't be tried to that extent).

Like I said in my other email, I think having another fork of the Lustre
tree, especially one that is starting from two-year-old code is likely to
fail, because there will be twice as much effort spent to maintain the two
trees.  I'd rather see the cleanups and features go hand-in-hand into the
same tree.  I'd be thrilled to have more reviews done on the features before
they are landed, but we can't just stop all feature development for a year
or two (or five) while the code is merged into the upstream kernel.

>> There are some very big (as in code size) features missing from
>> upstream.  For example, Multi-Rail.  When should that be pushed
>> relative to code cleanups?
>
> Never add features to ugly code - fix the code first.
> The doesn't mean you cannot add any feature to lustre until all of
> lustre is beautiful.  But it does mean that if I can see in a patch some
> ugly code and a new feature, then I won't be happy.  First clean up just
> enough of the ugliness so that it won't be visible in the patch that
> adds the feature.

The issue is that we _can't_ just stop the development of new code/features
for such a long time.  There are huge supercomputers being deployed or in
planning that depend on these new features, or they wouldn't have been
developed in the first place.

Consider if the NFSv4 spec was written and the code was developed, and you
were told you needed to go back to NFSv2 and start again?

> But again, this is getting a bit too hypothetical.   If you care about a
> feature, then post a patch.  We can take it from there.  The fact that
> you care enough to post a patch cares significant weight - a lot more
> weight than just asking about some feature.

We definitely aren't at the point of "asking for some feature to be developed".
At a minimum the starting point of the new upstream code needs to be the
current release, or any resources that could possibly be put towards improving
the Lustre code would be squandered on porting all of those patches to the
upstream tree.  I'm fine with spending time to improve the code that exists
today, but lets not start with a huge deficit from the outset.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel Corporation







_______________________________________________
lustre-devel mailing list
lustre-devel at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180607/9e4d04a7/attachment-0001.html>

[lustre-devel] A new drivers/staging/lustre

[NeilBrown]

On Thu, Jun 07 2018, Dilger, Andreas wrote:

> On Jun 7, 2018, at 15:38, NeilBrown <neilb@suse.com> wrote:
>> 
>> On Thu, Jun 07 2018, Doug Oucharek wrote:
>> 
>>> What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:
>>> 
>>> 
>>>  *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
>>>  *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).
>>> 
>>> For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.
>> 
>> My (undoubtedly biased) perspective on the history of lustre in staging
>> goes like this:
>> There are two things needed for some out-of-tree code to get into
>> mainline Linux:  the code needs to be integrated and the community needs
>> to be integrated (or a new sub-community needs to form).
>> In the case of lustre, the code was never really integrated because the
>> community never really tried to integrate.
>
> One of the issues here was that the group (not Intel) that submitted the
> Lustre code to the staging tree promptly abandoned it for a couple of
> years after they submitted it upstream, after promising the community
> that they were in it for the long run.  That put the upstream integration
> behind the eight-ball from the start.

Yes, we have a lot of "technical debt" to recover from, or repay, or
what ever analogy works best.

>
>>  Integrating and becoming
>> part of the Linux community takes time and effort, and it is quite
>> possible that management for various developers didn't allocate enough
>> time over a long enough period.  Integrating also requires a change in
>> attitude and I don't see much evidence of that.  I see clear evidence of
>> an "us and them" attitude among (some) lustre developers - almost as
>> though upstream linux is hostile territory full of unfriendly developers
>
> Ah, but it *is* hostile territory, if you are not among the "in crowd".
> Christoph can get any change he wants to be accepted, but if someone else
> tries to push something similar it can be rejected outright or ignored
> for months or years.

Christoph is taken more seriously than others primarily because he is
more consistently right.  He has built up trust with other key people
over years.  You cannot expect to get the same returns without the same
investment.
If he has interests that overlap with your, then it is certainly worth
trying to understand his point of view, and worth trying to work with
him.  That can be a challenge as his style can be a bit abrasive, but
I'm sure it can yield good returns.

Christoph may seem like the enemy, but he is really just another
developer like you or me.  He has his priorities and his hang-ups and he
wants to get his work done and to keep Linux generally in a healthy
state.
If you think of him (or anyone else) as the enemy, it will poison your
relationship before you even start.
If you think of him as a future friend who speaks a different language
which it will take you a while to understand, I think you will make more
progress.
Importantly, if you appear to be trying to work together you will create
a good impression on others and start you build your social capital, and
start to earn trust.

>
>> who always reject our excellent code (even though they have lots of
>> horrible code themselves).  *We* need to see ourselves as part of the
>> Linux community, and we need to care about all of Linux as though it was
>> all ours (it *is* all ours, but *we* are a much larger group now).
>> 
>> Yes, the current code needs to be improved, bugs need to be fixed, and
>> features need to be added.  The order in which these is done is not the
>> most important things - if it were, Greg would have never accepted any
>> new features.  However he *did* accept them, but tried to remind the
>> lustre developers that there was other work to do.
>> 
>> Working together in one (single) community requires give-and-take.
>> Greg's behaviour as just described seems to be evidence of
>> give-and-take.  I think he kicked lustre out of staging because he
>> concluded that he was never going to get the matching give-and-take in
>> return.
>> 
>> So to answer your opening question, my focus for this tree is to train
>> any lustre developers who wish to engage about how to be part of the
>> Linux community.  As I've already said - I will accept features but I
>> prefer cleanups first.  I don't want to try to explain further than that
>> because it will be too hypothetical and unhelpful.  We - the Linux
>> community - don't work in hypotheticals.  We work with concrete objects
>> like patches.  So send me a patch and I will tell you what I think of
>> that specific patch.  It is up to you to generalise what I say to other
>> patches.  It might also be up to you to argue your case and tell me why
>> I'm wrong.  I'll be patient (because good upstream maintainers are) but
>> patience doesn't last forever (for Greg, it lasted about 5 years - I
>> hope mine won't be tried to that extent).
>
> Like I said in my other email, I think having another fork of the Lustre
> tree, especially one that is starting from two-year-old code is likely to
> fail, because there will be twice as much effort spent to maintain the two
> trees.  I'd rather see the cleanups and features go hand-in-hand into the
> same tree.  I'd be thrilled to have more reviews done on the features before
> they are landed, but we can't just stop all feature development for a year
> or two (or five) while the code is merged into the upstream kernel.

Where did this idea of stopping all feature development come from?  It
is a ridiculous idea.
Of course, getting the code into shape for upstream inclusion will
require effort and time and someone will have to do that.  While they
are doing it they won't be working on features.
That either means new developers will need to get involved (like me) or
current developers will need to dedicate some of their time to the
upstream work.
So feature development doesn't have to stop, but it might slow unless
extra developers are added.

There seem to be two options:
1/ code clean up and continuing feature development happen in the one
   tree.
2/ feature development happens in one tree while code cleanup and
   integration of newly developed feature happens in another tree.

There are two variation on option 2.
2a/ the code-cleanup tree is based on what is being removed from
    drivers/staging.
2b/ the code-cleanup tree is created as a new fork of the current
    devel tree.

Each of 2a and 2b would have costs and benefits that the other doesn't
have.
I like 2a because a lot of cleanup has already been done, and because
porting patches across from the devel tree will provide a good
opportunity to review those patches.  Reviewing a large body of existing code
is really hard.  Reviewing patches one at a time is easier as they tend
to be conceptually coherent.

I like 2a and have already made steps in that direction.  But the
decision needs to be made by the people who will be doing the work.
Hopefully that isn't just me.

So: who has time to commit to creating an upstream-able version of
  lustre?  If everyone who will be committing time could make a clear
  statement of how they would like to proceed, then I'm sure we can come
  to agreement.

>
>>> There are some very big (as in code size) features missing from
>>> upstream.  For example, Multi-Rail.  When should that be pushed
>>> relative to code cleanups?
>> 
>> Never add features to ugly code - fix the code first.
>> The doesn't mean you cannot add any feature to lustre until all of
>> lustre is beautiful.  But it does mean that if I can see in a patch some
>> ugly code and a new feature, then I won't be happy.  First clean up just
>> enough of the ugliness so that it won't be visible in the patch that
>> adds the feature.
>
> The issue is that we _can't_ just stop the development of new code/features
> for such a long time.  There are huge supercomputers being deployed or in
> planning that depend on these new features, or they wouldn't have been
> developed in the first place.
>
> Consider if the NFSv4 spec was written and the code was developed, and you
> were told you needed to go back to NFSv2 and start again?
>
>> But again, this is getting a bit too hypothetical.   If you care about a
>> feature, then post a patch.  We can take it from there.  The fact that
>> you care enough to post a patch cares significant weight - a lot more
>> weight than just asking about some feature.
>
> We definitely aren't at the point of "asking for some feature to be developed".
> At a minimum the starting point of the new upstream code needs to be the
> current release, or any resources that could possibly be put towards improving
> the Lustre code would be squandered on porting all of those patches to the
> upstream tree.  I'm fine with spending time to improve the code that exists
> today, but lets not start with a huge deficit from the outset.

We have a huge deficit either way.  One way we have a deficit of
functionality, the other way we have a deficit of quality.

But let's not spend too much time talking about the problems.  I'd much
rather hear what you are doing about it.  If other people are working,
I'll align with them.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180608/8b8fe4ad/attachment.sig>

[lustre-devel] A new drivers/staging/lustre

[NeilBrown]

On Thu, Jun 07 2018, Dilger, Andreas wrote:

> On Jun 7, 2018, at 16:53, Doug Oucharek <doucharek@cray.com> wrote:
>> 
>> Does this mean we would be pushing patches to Gerrit rather than having to email them out?  I believe for the test system to work automatically, the answer is yes.
>
> I'm not an expert with Gerrit configuration, so I'm not sure if there is an
> email-to-change gateway plugin for it or not.  I believe it can be configured
> to send the full patch in emails.  I get an email for every patch submitted,
> but I configure it to only deliver the commit message and not the whole patch.
>
> Since everyone is using Git anyway, using "git push" to submit patches to Gerrit
> for review (and forwards them to the list, if that is what people want) isn't
> more effort than "git send-email" to send them to the list directly.

I think that pushing patches to Gerrit for testing is essential.  I'm
less keen on using it for review because it doesn't fit my current
workflow, but I could probably adjust if necessary.  My preference would
be for people to develop patches doing their own local testing to avoid
the really embarrassing errors (I have a 4-node virtual cluster on my
desktop) and then post them to a list for wider review.  If that passes
then someone (maybe one of a small team other something) adds them to a
git tree (either with "git am" or "git pull", whatever works for people)
which gerrit runs tests on.  We would try to get this tree incorporated
into the 01.org testing so that we get even more coverage.  They might
not be able to run lustre-fs tests, but they do static analysis testing
which should cover lustre.

Code wouldn't migrate to the "main" tree until it had passed both review
and testing.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180608/a6a71e21/attachment.sig>

[lustre-devel] A new drivers/staging/lustre

[NeilBrown]

On Thu, Jun 07 2018, Dilger, Andreas wrote:

> On Jun 6, 2018, at 20:29, NeilBrown <neilb@suse.com> wrote:
>> Greg's patch to remove lustre has now landed in this staging-next tree,
>> so I suspect it will get to Linus before too long.  So I have to find a
>> new place to work on lustre.
>> 
>> I've added 2 branches to
>>   git://git.neil.brown.name/linux
>> 
>> lustre:
>>   is based on Greg's patch that removes lustre, and starts with a
>>   revert of the patch, followed by a merge of v4.17.
>>   I plan to merge each release and RC from Linus, and also
>>   add lustre patches that I think are "ready".  That will usually mean
>>   they have been posted to this list at least a week earlier, and
>>   have not had a credible negative response (Acks and Reviewed-by
>>   would be nice).
>>   I plan to update this branch about once a week, and to never rebase
>>   it.
>> 
>> lustre-testing:
>>   is based on 'lustre' and has most of my current lustre-related work.
>>   It includes assorted patches that are not specifically for lustre
>>   (rhashtables mostly at the moment).  Patches will move from here
>>   to 'lustre' or to mainline when they are ready.
>>   I plan to update this branch on most days that I work on Lustre,
>>   and expect it to rebase frequently.
>
> We also have a clone of Greg's staging branch on our Gerrit server,
> which could be replaced with a "vanilla" upstream kernel clone that
> holds a copy of the for-upstream Lustre tree.
>
> One benefit of hosting the tree on the Gerrit server is that it is
> much easier to get automated testing of the patches (a lot or a little)
> to ensure that the code doesn't break anything obvious.
>
>> I'm happy to review and, if acceptable, apply patches from other
>> developers.  I have fairly high standards, but if I don't accept your
>> patch I'll explain why and possible help fix it.
>> I'm happy to accept enhancements and new features, but these need
>> to be of a quality that would be accepted upstream.
>> I'm only interested in client-side code at present - nothing that is
>> only used on the server.  I do want to include server-side eventually,
>> but I need some focus for now.
>> 
>> I hope to get to a stage where the code is of suitable quality that I
>> can submit it to Linux as a new filesystem.  I hope that will happen
>> this year.
>
> I think one major drawback of starting with the from-staging version of
> the code is that this is significantly behind the out-of-tree master
> code and would need a lot of effort to catch up.
>
> I think the only way we are going to make this work is if the cleanups
> go into the same repo as the ongoing development.  Otherwise, we'll be
> back in the same situation as we are now where there are two different
> versions of the code and we need to move patches back and forth between
> them.
>
> On the plus side, this will avoid code drift, and the extra efforts of
> porting patches back and forth.  On the minus side, it means (on some
> fronts) that the code will regress from what is upstream, because we
> don't have all of the trivial whitespace cleanups and other drive-by
> kernel newbie patches that went into the upstream client.  However, it
> does mean that those cleanups would now become part of the single code
> tree and not be lost again in the future.
>
> This would also mean some restructuring of the current Lustre tree so
> that it could build as a drop-in at linux/fs/lustre but that is probably
> a good idea for the long term in any case.

One of the difficulties I imagine with a single tree is that the current
tree has lots of backward compatibility support to allow the code to
compile on old kernels.  Having to maintain that would seem to interfere
with getting the code into an upstreamable state.
If the lustre development model could change to be working primarily in
a tree what was kept in-sync with upstream and which didn't carry
back-compat code, then I think we might be able to do development and
cleanup in the same tree.
Every other linux project (that I know of) works this way.  Patches are
developed in an upstream kernel, then back-ported to other kernels as
needed.  Enterprise distros do this all the time and are quite good at
it.

Can we move all the kernel code out of the lustre tree and into a clone
of Linux??

Thanks,
NeilBrown


>
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180608/da30e556/attachment.sig>

[lustre-devel] A new drivers/staging/lustre

[James Simmons]

> > On Thu, Jun 07 2018, Doug Oucharek wrote:
> > 
> >> What is the focus of landings in this tree?  There are two things needing to be done for an upstream Lustre:
> >> 
> >> 
> >>  *   Get the source code to meet the Linux guidelines so it is acceptable to be in mainline.
> >>  *   Get the binary product to have all the features and bug fixes that are in the Intel community tree so end users are interested in using the upstream version (users are unlikely to use a version of Lustre which is not current).
> >> 
> >> For the now-deleted staging area, we were supposed to be focusing on the first item but were submitting patches for the second item (syncing with Intel tree).  In my opinion, this is the core reason for never being able to get out of staging and getting deleted.
> > 
> > My (undoubtedly biased) perspective on the history of lustre in staging
> > goes like this:
> > There are two things needed for some out-of-tree code to get into
> > mainline Linux:  the code needs to be integrated and the community needs
> > to be integrated (or a new sub-community needs to form).
> > In the case of lustre, the code was never really integrated because the
> > community never really tried to integrate.
> 
> One of the issues here was that the group (not Intel) that submitted the
> Lustre code to the staging tree promptly abandoned it for a couple of
> years after they submitted it upstream, after promising the community
> that they were in it for the long run.  That put the upstream integration
> behind the eight-ball from the start.

The leason from this is that the upstream efforts can not be placed on an
single organization's efforts. It has to be a group effort. Before Neil
showed up I was the only person working on the upstream efforts for some
time. I came under pressure to abandon this effort but since I do this
on the weekends in my free time it doesn't impact my normal job duties.
So I managed to get away with this work. It does help that actually
useful by products from this effort shows up in the general lustre
community which in turn benefits my employeer. I bet for EMC the burden
was to much and since no one supported them they abandon this work.
The main reason people abandon things in life is feeling unsupported.
Even if all you do is act as a cheerleader it can have a big impact
on the people doing the work. At one time I felt the same way with the 
upstream client which I shared with Peter Jones. Peter pointed out that he 
knows sites that do use the upstream client. Since then my focus has been 
aimed at supporting those users. With my focus is on the users so if 
companies join or don't join in is not important to me.

> > Yes, the current code needs to be improved, bugs need to be fixed, and
> > features need to be added.  The order in which these is done is not the
> > most important things - if it were, Greg would have never accepted any
> > new features.  However he *did* accept them, but tried to remind the
> > lustre developers that there was other work to do.
> > 
> > Working together in one (single) community requires give-and-take.
> > Greg's behaviour as just described seems to be evidence of
> > give-and-take.  I think he kicked lustre out of staging because he
> > concluded that he was never going to get the matching give-and-take in
> > return.

Based on the emails sent I see no one understood the work I was doing for 
the last 2 years. I guess that is my fault for not having general 
discussion emails on the efforts. Mind you at first I tried to send 
non patch emails to the staging mailing list but they always were 
filtered out so I gave up. Also no one every talked to me about what the
cleanups should be. So I have always worked on what I know is a problem.
Mainly things reported by users and also things reported back to Oleg
from the VFS developers. That is how the roadmap came into being that
I discussed at developer's day for LUG.

First of all the upstream client is NOT at a 2.4 version. Its close to 
a 2.9 version. In fact its newer then the default lustre client Cray ships
with (2.7). Second I haven't pushed features for over a year. The first
year I did spend syncing the tree up to 2.8+ version which got us to a
point where people were willing to try it. 

After that it became this constant smash the bugs for the last year. The 
port to sysfs and the 64 bit time work was incorrect and lead to all kinds 
of regressions. Also the xattr which lustre heavly depends on was broken. 
Nearly all xattr bugs are fixed. I nearly have resolved most of the 64
bit time and sysfs bugs. Still haven't push them yet but that is next. 
So my focus is reduce all the regression to a manageable level. I really
didn't want to see a lustre client out of staging that was a complete 
piece of 'bleep'.

The client feature wise is nearly 2.9 but many bug fixes from 2.10 and
above have been merged to the upstream code. The gap is much smaller than
you realize. So the upstream client is strange hybrid. 

> > So to answer your opening question, my focus for this tree is to train
> > any lustre developers who wish to engage about how to be part of the
> > Linux community.  As I've already said - I will accept features but I
> > prefer cleanups first.  I don't want to try to explain further than that
> > because it will be too hypothetical and unhelpful.  We - the Linux
> > community - don't work in hypotheticals.  We work with concrete objects
> > like patches.  So send me a patch and I will tell you what I think of
> > that specific patch.  It is up to you to generalise what I say to other
> > patches.  It might also be up to you to argue your case and tell me why
> > I'm wrong.  I'll be patient (because good upstream maintainers are) but
> > patience doesn't last forever (for Greg, it lasted about 5 years - I
> > hope mine won't be tried to that extent).
> 
> Like I said in my other email, I think having another fork of the Lustre
> tree, especially one that is starting from two-year-old code is likely to
> fail, because there will be twice as much effort spent to maintain the two
> trees.  I'd rather see the cleanups and features go hand-in-hand into the
> same tree.  I'd be thrilled to have more reviews done on the features before
> they are landed, but we can't just stop all feature development for a year
> or two (or five) while the code is merged into the upstream kernel.

I don't think having another tree is such a big issue. As I pointed out
earlier Cray carries their own lustre client. Also Livermore Labs 
has their own special Choas lustre tree. Lastly we do backports to LTS
version of lustre all the time.

I'm really against using the out of tree for this kernel cleanup for a
few reasons. The gap between upstream and master is less than you think.
Outside the features landed the majority of the patches landed to the 
OpenSFS/Intel branch is my work to sync both the upstream client and the 
Intel community branch. Next from personal experinece it takes a very very
long time to land cleanup patches for the Intel branch. It normally takes
3 release cycles to complete any work which means we are looking at least
1.5 years to finish meeting upstream requirements. Now going in the other
direction in a few months we could reach parity with the out of tree
source. Especially since the gap is much smaller now. I ready to close
the gap.

[lustre-devel] A new drivers/staging/lustre

[James Simmons]

> Greg's patch to remove lustre has now landed in this staging-next tree,
> so I suspect it will get to Linus before too long.  So I have to find a
> new place to work on lustre.
> 
> I've added 2 branches to
>    git://git.neil.brown.name/linux

Its seems to be down. Do you plan to move the tree somewhere else. Perhaps
your github page?

[lustre-devel] A new drivers/staging/lustre

[James Simmons]

> >> lustre-testing:
> >>    is based on 'lustre' and has most of my current lustre-related work.
> >>    It includes assorted patches that are not specifically for lustre
> >>    (rhashtables mostly at the moment).  Patches will move from here
> >>    to 'lustre' or to mainline when they are ready.
> >>    I plan to update this branch on most days that I work on Lustre,
> >>    and expect it to rebase frequently.
> >
> > I had question about that. Some things in Lustre could in theory be merged
> > into the linux kernel proper. Can that be done still?
> 
> What things?
> 
> If it measurably benefits the kernel proper, then I suspect it might be
> worth submitting.  Things can go direct without going though staging -
> they just have to be of good quality with good justification (and
> sometimes lots of patience).

One piece of work done for Lustre was the ability to submit "units" like
K, M, G for sizes to sysfs files. This was developed before string helpers
was created and the code for Lustre is well very ugly. So I rewrote it
in the style of string helpers and it can be handly for other things as 
well such as module parameters setting. Lustre does have certain features
in it debugging system that can applied to the linux kernel debugging
system. That is more down the road. Their are few others like the crypto
wrapper for alder. I don't see why that can't be mainstream.
 
> >  
> >> I'm happy to review and, if acceptable, apply patches from other
> >> developers.  I have fairly high standards, but if I don't accept your
> >> patch I'll explain why and possible help fix it.
> >
> > Also long as you talk to me :-) I'm an easy person to work with. If I 
> > refuse a patch with do the same. I might sometimes seem irrational 
> > but I have valid reasons. Well at least in my head.
> >
> > We need to really layout the roadmap.
> 
> I have very little faith in road maps - I prefer to make steps.  Once we
> have made all the steps, we can look back and see what the map looked
> like in retrospect.

I was looking to see what you plan to work on. I wanted to avoid 
duplicating work. Some things I know of done already are listed under:

https://jira.hpdd.intel.com/browse/LU-9855
https://jira.hpdd.intel.com/browse/LU-10257
https://jira.hpdd.intel.com/browse/LU-10994

Their are a few others I have to hunt down but these are the major ones.

[lustre-devel] A new drivers/staging/lustre

[NeilBrown]

On Tue, Jun 12 2018, James Simmons wrote:

>> Greg's patch to remove lustre has now landed in this staging-next tree,
>> so I suspect it will get to Linus before too long.  So I have to find a
>> new place to work on lustre.
>> 
>> I've added 2 branches to
>>    git://git.neil.brown.name/linux
>
> Its seems to be down. Do you plan to move the tree somewhere else. Perhaps
> your github page?

Bother - that's what you get for using Linux to host things (my VM
doesn't have heaps of memory and git likes to use a lot, and doesn't
always clean up well after itself).

git.neil.brown.name is back, and github.com/neilbrown/linux now
has two new branches: lustre/lutre and lustre/lustre-testing

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180612/2c8ebe9b/attachment.sig>

[lustre-devel] A new drivers/staging/lustre

[NeilBrown]

On Tue, Jun 12 2018, James Simmons wrote:

>> >> lustre-testing:
>> >>    is based on 'lustre' and has most of my current lustre-related work.
>> >>    It includes assorted patches that are not specifically for lustre
>> >>    (rhashtables mostly at the moment).  Patches will move from here
>> >>    to 'lustre' or to mainline when they are ready.
>> >>    I plan to update this branch on most days that I work on Lustre,
>> >>    and expect it to rebase frequently.
>> >
>> > I had question about that. Some things in Lustre could in theory be merged
>> > into the linux kernel proper. Can that be done still?
>> 
>> What things?
>> 
>> If it measurably benefits the kernel proper, then I suspect it might be
>> worth submitting.  Things can go direct without going though staging -
>> they just have to be of good quality with good justification (and
>> sometimes lots of patience).
>
> One piece of work done for Lustre was the ability to submit "units" like
> K, M, G for sizes to sysfs files. This was developed before string helpers
> was created and the code for Lustre is well very ugly. So I rewrote it
> in the style of string helpers and it can be handly for other things as 
> well such as module parameters setting.

I cannot work out what we might need beyond string_helpers, and I cannot
find any code in lustre that might be what you are referring to.  But I
agree that making it easy to use unit in sysfs is probably a generally
good idea.

>                                         Lustre does have certain features
> in it debugging system that can applied to the linux kernel debugging
> system. That is more down the road. Their are few others like the crypto
> wrapper for alder. I don't see why that can't be mainstream.

I have wondered why crypto-adler isn't in the main crypto library.  I
understand that it is weak and deprecated, but I doubt that is a reason
to keep the code out.  Maybe we should just try to submit it.

>  
>> >  
>> >> I'm happy to review and, if acceptable, apply patches from other
>> >> developers.  I have fairly high standards, but if I don't accept your
>> >> patch I'll explain why and possible help fix it.
>> >
>> > Also long as you talk to me :-) I'm an easy person to work with. If I 
>> > refuse a patch with do the same. I might sometimes seem irrational 
>> > but I have valid reasons. Well at least in my head.
>> >
>> > We need to really layout the roadmap.
>> 
>> I have very little faith in road maps - I prefer to make steps.  Once we
>> have made all the steps, we can look back and see what the map looked
>> like in retrospect.
>
> I was looking to see what you plan to work on. I wanted to avoid 
> duplicating work. Some things I know of done already are listed under:

Providing that we regularly talk about what we are doing I think the
chance of duplication is fairly low, and I don't think the cost of
duplication is very high.  If we've both worked of doing X, then we
should each be able to review the work of the other quite quickly.

My current focus is cleaning up the build (reducing modules and
exports), renaming files (removing 'linux' from various names),
and removing unused cruft from include files (files named *compat.h are
next).
I then want to gain a more complete understanding of LNet.

>
> https://jira.hpdd.intel.com/browse/LU-9855

I had thought about attacking this, but it isn't on my current
shortlist.  I'll let you know if it gets near the top.

> https://jira.hpdd.intel.com/browse/LU-10257

I don't really know what this is about.  I think it is a much deeper
clean-up than the superficial stuff I've been focusing on.
If I read correctly, this has already been addressed in legacy-lustre.
In that case I probably would only get to it when I start going through
the git log locking for things to migrate to linux-lustre.

> https://jira.hpdd.intel.com/browse/LU-10994

This is even less on my radar than the above.

I suggest you feel free to do whatever seems suitable without worrying
if I'm already working there - but do report results and request review
often, even if you aren't completely happy yet.
If I ever find you working on something that I've started, I'll quietly
hide my work and just use the knowledge gained to provide insightful
review of you work.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180612/25c68dd0/attachment.sig>

[lustre-devel] A new drivers/staging/lustre

[NeilBrown]

On Tue, Jun 12 2018, James Simmons wrote:

> The main reason people abandon things in life is feeling unsupported.
> Even if all you do is act as a cheerleader it can have a big impact
> on the people doing the work.

So much "yes".
Receiving feed-back, encouragement, affirmation that the work is valued,
is a REALLY big thing.  I do this because I'm paid, but also because I
enjoy the challenge and I value the community involvement.
A colleague in SUSE recently forwarded a line from an IRC conversation
that he was part of where someone commended me on my work on Lustre.  It
was really nice to receive that (though maybe a shame that it didn't
come more directly).
Who-ever said that - thanks!  You don't have to be writing patches to be
supporting this work.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180612/2d5cd6e0/attachment.sig>

[lustre-devel] A new drivers/staging/lustre

[James Simmons]

> 
> >> Greg's patch to remove lustre has now landed in this staging-next tree,
> >> so I suspect it will get to Linus before too long.  So I have to find a
> >> new place to work on lustre.
> >> 
> >> I've added 2 branches to
> >>    git://git.neil.brown.name/linux
> >
> > Its seems to be down. Do you plan to move the tree somewhere else. Perhaps
> > your github page?
> 
> Bother - that's what you get for using Linux to host things (my VM
> doesn't have heaps of memory and git likes to use a lot, and doesn't
> always clean up well after itself).
> 
> git.neil.brown.name is back, and github.com/neilbrown/linux now
> has two new branches: lustre/lutre and lustre/lustre-testing

It lives!!! Trying to grab the tree from git.neil.brown.name and still 
chokes but the github tree works works like a charm. I built and tested
the tree and it works.

[lustre-devel] A new drivers/staging/lustre

[NeilBrown]

On Wed, Jun 13 2018, James Simmons wrote:

>> 
>> >> Greg's patch to remove lustre has now landed in this staging-next tree,
>> >> so I suspect it will get to Linus before too long.  So I have to find a
>> >> new place to work on lustre.
>> >> 
>> >> I've added 2 branches to
>> >>    git://git.neil.brown.name/linux
>> >
>> > Its seems to be down. Do you plan to move the tree somewhere else. Perhaps
>> > your github page?
>> 
>> Bother - that's what you get for using Linux to host things (my VM
>> doesn't have heaps of memory and git likes to use a lot, and doesn't
>> always clean up well after itself).
>> 
>> git.neil.brown.name is back, and github.com/neilbrown/linux now
>> has two new branches: lustre/lutre and lustre/lustre-testing
>
> It lives!!! Trying to grab the tree from git.neil.brown.name and still 
> chokes but the github tree works works like a charm. I built and tested
> the tree and it works.

I guess I'm going to have to deprecate using neil.brown.name for git, or
else arrange for a lot more memory. linux.git is getting too big.
It probably works OK if you "git clone --reference=/some/local/linux/tree/.git"
but people don't think to do that.
I'll make sure to keep github up to date.

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180614/11f91e72/attachment.sig>