RE: [PATCH] staging: exfat: add exfat filesystem code to

RE: [PATCH] staging: exfat: add exfat filesystem code to

[Namjae Jeon]

We are excited to see this happening and would like to state that we appreciate
time and
effort which people put into upstreaming exfat. Thank you!

However, if possible, can we step back a little bit and re-consider it? We
would prefer to
see upstream the code which we are currently using in our products - sdfat - as
this can
be mutually benefitial from various points of view.

Thanks!

> ---------- Forwarded message ---------
> 보낸사람: Ju Hyung Park <qkrwngud825@gmail.com>
> Date: 2019년 9월 16일 (월) 오전 3:49
> Subject: Re: [PATCH] staging: exfat: add exfat filesystem code to
> To: Greg KH <gregkh@linuxfoundation.org>
> Cc: <alexander.levin@microsoft.com>, <devel@driverdev.osuosl.org>, <linux-
> fsdevel@vger.kernel.org>, <linux-kernel@vger.kernel.org>, Valdis Kletnieks
> <valdis.kletnieks@vt.edu>
> 
> 
> Hi Greg,
> 
> On Sun, Sep 15, 2019 at 10:54 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> > Note, this just showed up publically on August 12, where were you with
> > all of this new code before then?  :)
> 
> My sdFAT port, exfat-nofuse and the one on the staging tree, were all
> made by Samsung.
> And unless you guys had a chance to talk to Samsung developers
> directly, those all share the same faith of lacking proper development
> history.
> 
> The source I used was from http://opensource.samsung.com, which
> provides kernel sources as tar.gz files.
> There is no code history available.
> 
> > For the in-kernel code, we would have to rip out all of the work you did
> > for all older kernels, so that's a non-starter right there.
> 
> I'm aware.
> I'm just letting mainline know that there is potentially another (much
> better) base that could be upstreamed.
> 
> If you want me to rip out older kernel support for upstreaming, I'm
> more than happy to do so.
> 
> > As for what codebase to work off of, I don't want to say it is too late,
> > but really, this shows up from nowhere and we had to pick something so
> > we found the best we could at that point in time.
> 
> To be honest, whole public exFAT sources are all from nowhere unless
> you had internal access to Samsung's development archive.
> The one in the current staging tree isn't any better.
> 
> I'm not even sure where the staging driver is from, actually.
> 
> Samsung used the 1.2.x versioning until they switched to a new code
> base - sdFAT.
> The one in the staging tree is marked version 1.3.0(exfat_super.c).
> I failed to find anything 1.3.x from Samsung's public kernel sources.
> 
> The last time exFAT 1.2.x was used was in Galaxy S7(released in 2016).
> Mine was originally based on sdFAT 2.1.10, used in Galaxy S10(released
> in March 2019) and it just got updated to 2.2.0, used in Galaxy
> Note10(released in August 2019).
> 
> > Is there anything specific in the codebase you have now, that is lacking
> > in the in-kernel code?  Old-kernel-support doesn't count here, as we
> > don't care about that as it is not applicable.  But functionality does
> > matter, what has been added here that we can make use of?
> 
> This is more of a suggestion of
> "Let's base on a *much more recent* snapshot for the community to work on",
> since the current one on the staging tree also lacks development history.
> 
> The diff is way too big to even start understanding the difference.
> 
> 
> With that said though, I do have some vague but real reason as to why
> sdFAT base is better.
> 
> With some major Android vendors showing interests in supporting exFAT,
> Motorola notably published their work on public Git repository with
> full development history(the only vendor to do this that I'm aware
> of).
> Commits like this:
> https://github.com/MotorolaMobilityLLC/kernel-msm/commit/7ab1657 is
> not merged to exFAT(including the current staging tree one) while it
> did for sdFAT.
> 
> 
> The only thing I regret is not working on porting sdFAT sooner.
> I definitely didn't anticipate Microsoft to suddenly lift legal issues
> on upstreaming exFAT just around when I happen to gain interest in
> porting sdFAT.
> 
> If my port happened sooner, it would have been a no-brainer for it to
> be considered as a top candidate for upstreaming.
> 
> > And do you have any "real" development history to look at instead of the
> > "one giant commit" of the initial code drop?  That is where we could
> > actually learn what has changed over time.  Your repo as-is shows none
> > of the interesting bits :(
> 
> As I mentioned, development history is unobtainable, even for the
> current staging tree or exfat-nofuse.
> (If you guys took exfat-nofuse, you can also see that there's barely
> any real exFAT-related development done in that tree. Everything is
> basically fixes for newer kernel versions.)
> 
> The best I could do, if someone's interested, is to diff all versions
> of exFAT/sdFAT throughout the Samsung's kernel versions, but that
> still won't give us reasons as to why the changes were made.
> 
> TL;DR
> My suggestion - Let's base on a much newer driver that's matured more,
> contains more fixes, gives (slightly?) better performance and
> hopefully has better code quality.
> 
> Both drivers are horrible.
> You said it yourself(for the current staging one), and even for my new
> sdFAT-base proposal, I'm definitely not comfortable seeing this kind
> of crap in mainline:
> https://github.com/arter97/exfat-linux/commit/0f1ddde
> 
> However, it's clear to me that the sdFAT base is less-horrible.
> 
> Please let me know what you think.
> 
> > thanks,
> >
> > greg kh
> 
> Thanks.
> 



_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Valdis Klētnieks]

[-- Attachment #1.1: Type: text/plain, Size: 1247 bytes --]

On Tue, 17 Sep 2019 12:02:01 +0900, "Namjae Jeon" said:
> We are excited to see this happening and would like to state that we appreciate time and
> effort which people put into upstreaming exfat. Thank you!

The hard part - getting Microsoft to OK merging an exfat driver - is done.

All we need now is to get a driver cleaned up. :)

> However, if possible, can we step back a little bit and re-consider it? We would prefer to
> see upstream the code which we are currently using in our products - sdfat - as
> this can be mutually benefitial from various points of view.

I'm working off a somewhat cleaned up copy of Samsung's original driver,
because that's what I had knowledge of.  If the sdfat driver is closer to being
mergeable, I'd not object if that got merged instead.

But here's the problem... Samsung has their internal sdfat code, Park Yu Hyung
has what appears to be a fork of that code from some point (and it's unclear ,
and it's unclear which one has had more bugfixes and cleanups to get it to
somewhere near mainline mergeable.

Can you provide a pointer to what Samsung is *currently* using? We probably
need to stop and actually look at the code bases and see what's in the best
shape currently.


[-- Attachment #1.2: Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 169 bytes --]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

['Greg KH']

On Tue, Sep 17, 2019 at 12:02:01PM +0900, Namjae Jeon wrote:
> We are excited to see this happening and would like to state that we appreciate
> time and
> effort which people put into upstreaming exfat. Thank you!
> 
> However, if possible, can we step back a little bit and re-consider it? We
> would prefer to see upstream the code which we are currently using in
> our products - sdfat - as this can be mutually benefitial from various
> points of view.

What is "different" in it from the code that is currently in linux-next?
What is supported "better"?  The code we have today seems to work for
people, so what are we missing here?

Also, submitting patches for this codebase to bring it up to the level
of functionality you need would be wonderful, can you do that?

thanks,

greg k-h
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Gao Xiang]

Hi,

On Tue, Sep 17, 2019 at 12:02:01PM +0900, Namjae Jeon wrote:
> We are excited to see this happening and would like to state that we appreciate
> time and
> effort which people put into upstreaming exfat. Thank you!
> 
> However, if possible, can we step back a little bit and re-consider it? We
> would prefer to
> see upstream the code which we are currently using in our products - sdfat - as
> this can
> be mutually benefitial from various points of view.

(Only represent my personal views)

I'd like to know the detailed commit history as an individual Android hobbyist.

I noticed sdfat years ago and there is a difference from the previous exfat driver.

I have no idea it's a good way to blindly keep the code from some opensource tar
on some website. and so many forks on github (hard to know which one is more stable,
cleaner or latest)... someone could take more time and play a role in that actively
in the community and maybe draw a roadmap of this so I could study more and maybe
contribute a little in my spare time.

And I think if it permits, development on multiple branches could be avoided...
If I am wrong, please ignore me...

Thanks,
Gao Xiang

> 
> Thanks!
> 
> > ---------- Forwarded message ---------
> > ????????: Ju Hyung Park <qkrwngud825@gmail.com>
> > Date: 2019?? 9?? 16?? (??) ???? 3:49
> > Subject: Re: [PATCH] staging: exfat: add exfat filesystem code to
> > To: Greg KH <gregkh@linuxfoundation.org>
> > Cc: <alexander.levin@microsoft.com>, <devel@driverdev.osuosl.org>, <linux-
> > fsdevel@vger.kernel.org>, <linux-kernel@vger.kernel.org>, Valdis Kletnieks
> > <valdis.kletnieks@vt.edu>
> > 
> > 
> > Hi Greg,
> > 
> > On Sun, Sep 15, 2019 at 10:54 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> > > Note, this just showed up publically on August 12, where were you with
> > > all of this new code before then?  :)
> > 
> > My sdFAT port, exfat-nofuse and the one on the staging tree, were all
> > made by Samsung.
> > And unless you guys had a chance to talk to Samsung developers
> > directly, those all share the same faith of lacking proper development
> > history.
> > 
> > The source I used was from http://opensource.samsung.com, which
> > provides kernel sources as tar.gz files.
> > There is no code history available.
> > 
> > > For the in-kernel code, we would have to rip out all of the work you did
> > > for all older kernels, so that's a non-starter right there.
> > 
> > I'm aware.
> > I'm just letting mainline know that there is potentially another (much
> > better) base that could be upstreamed.
> > 
> > If you want me to rip out older kernel support for upstreaming, I'm
> > more than happy to do so.
> > 
> > > As for what codebase to work off of, I don't want to say it is too late,
> > > but really, this shows up from nowhere and we had to pick something so
> > > we found the best we could at that point in time.
> > 
> > To be honest, whole public exFAT sources are all from nowhere unless
> > you had internal access to Samsung's development archive.
> > The one in the current staging tree isn't any better.
> > 
> > I'm not even sure where the staging driver is from, actually.
> > 
> > Samsung used the 1.2.x versioning until they switched to a new code
> > base - sdFAT.
> > The one in the staging tree is marked version 1.3.0(exfat_super.c).
> > I failed to find anything 1.3.x from Samsung's public kernel sources.
> > 
> > The last time exFAT 1.2.x was used was in Galaxy S7(released in 2016).
> > Mine was originally based on sdFAT 2.1.10, used in Galaxy S10(released
> > in March 2019) and it just got updated to 2.2.0, used in Galaxy
> > Note10(released in August 2019).
> > 
> > > Is there anything specific in the codebase you have now, that is lacking
> > > in the in-kernel code?  Old-kernel-support doesn't count here, as we
> > > don't care about that as it is not applicable.  But functionality does
> > > matter, what has been added here that we can make use of?
> > 
> > This is more of a suggestion of
> > "Let's base on a *much more recent* snapshot for the community to work on",
> > since the current one on the staging tree also lacks development history.
> > 
> > The diff is way too big to even start understanding the difference.
> > 
> > 
> > With that said though, I do have some vague but real reason as to why
> > sdFAT base is better.
> > 
> > With some major Android vendors showing interests in supporting exFAT,
> > Motorola notably published their work on public Git repository with
> > full development history(the only vendor to do this that I'm aware
> > of).
> > Commits like this:
> > https://github.com/MotorolaMobilityLLC/kernel-msm/commit/7ab1657 is
> > not merged to exFAT(including the current staging tree one) while it
> > did for sdFAT.
> > 
> > 
> > The only thing I regret is not working on porting sdFAT sooner.
> > I definitely didn't anticipate Microsoft to suddenly lift legal issues
> > on upstreaming exFAT just around when I happen to gain interest in
> > porting sdFAT.
> > 
> > If my port happened sooner, it would have been a no-brainer for it to
> > be considered as a top candidate for upstreaming.
> > 
> > > And do you have any "real" development history to look at instead of the
> > > "one giant commit" of the initial code drop?  That is where we could
> > > actually learn what has changed over time.  Your repo as-is shows none
> > > of the interesting bits :(
> > 
> > As I mentioned, development history is unobtainable, even for the
> > current staging tree or exfat-nofuse.
> > (If you guys took exfat-nofuse, you can also see that there's barely
> > any real exFAT-related development done in that tree. Everything is
> > basically fixes for newer kernel versions.)
> > 
> > The best I could do, if someone's interested, is to diff all versions
> > of exFAT/sdFAT throughout the Samsung's kernel versions, but that
> > still won't give us reasons as to why the changes were made.
> > 
> > TL;DR
> > My suggestion - Let's base on a much newer driver that's matured more,
> > contains more fixes, gives (slightly?) better performance and
> > hopefully has better code quality.
> > 
> > Both drivers are horrible.
> > You said it yourself(for the current staging one), and even for my new
> > sdFAT-base proposal, I'm definitely not comfortable seeing this kind
> > of crap in mainline:
> > https://github.com/arter97/exfat-linux/commit/0f1ddde
> > 
> > However, it's clear to me that the sdFAT base is less-horrible.
> > 
> > Please let me know what you think.
> > 
> > > thanks,
> > >
> > > greg kh
> > 
> > Thanks.
> > 
> 
> 
> 
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Park Ju Hyung]

On Tue, 17 Sep 2019 00:19:36 -0400, "Valdis Klētnieks" said:
> I'm working off a somewhat cleaned up copy of Samsung's original driver,
> because that's what I had knowledge of.  If the sdfat driver is closer to being
> mergeable, I'd not object if that got merged instead.

Greg, as Valdis mentioned here, the staging tree driver is just another exFAT fork
from Samsung.
What's the point of using a much older driver?

sdFAT is clearly more matured and been put into more recent production softwares.
And as I wrote in previous email, it does include some real fixes.

As Namjae said too, Samsung would be more interested in merging sdFAT to upstream.
If we diverge, Samsung will have less reasons to contribute their patches to upstream.

Also, I think it makes much more sense to make Samsung the maintainer of this driver
(if they're willing to put in the manpower to do so). Asking them would be the first
step in doing so.

> But here's the problem... Samsung has their internal sdfat code, Park Yu Hyung
> has what appears to be a fork of that code from some point (and it's unclear ,
> and it's unclear which one has had more bugfixes and cleanups to get it to
> somewhere near mainline mergeable.

I made it extremely clear on where I took the code.

The initial commit: "sdfat: import from G973FXXU3ASG8" states which kernel source
I used.

You can simply search "G973FXXU3ASG8" on http://opensource.samsung.com and download
the source code. It'll match exactly with my initial commit.

My repository is basically rename + clean-up + older kernel compat.

I think we can all agree that using the sdFAT naming on non-Android is very
misleading, which is why I renamed it to exFAT.

sdFAT includes support for fat16/32, and as also mentioned in
https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/commit/?h=staging-next&id=58985a9d2d03e977db93bf574a16162766a318fe
this isn't desirable, especially in mainline.
I cleaned it up and removed some other Samsung's code that relies on proprietary
userspace tools such as defrag.

I believe my repository is in the cleanest state for getting merged to mainline,
compared to other drivers avilable out there.

If we happen to pick it to mainline, I think it'll also be quite trivial for Samsung
to pick mainline patches back to their sdFAT drivers used in Galaxy devices.

> Can you provide a pointer to what Samsung is *currently* using? We probably
> need to stop and actually look at the code bases and see what's in the best
> shape currently.

Namjae could probably elaborate here, but if I were to guess, there wasn't a
noticeable difference in recent sdFAT releases. Even the lastest Note10 kernel only
had some uevent changes.

I think the current latest public source's driver is the best one available.

Thanks.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Greg KH]

On Tue, Sep 17, 2019 at 02:31:34PM +0900, Park Ju Hyung wrote:
> On Tue, 17 Sep 2019 00:19:36 -0400, "Valdis Klētnieks" said:
> > I'm working off a somewhat cleaned up copy of Samsung's original driver,
> > because that's what I had knowledge of.  If the sdfat driver is closer to being
> > mergeable, I'd not object if that got merged instead.
> 
> Greg, as Valdis mentioned here, the staging tree driver is just another exFAT fork
> from Samsung.
> What's the point of using a much older driver?

It's the fact that it actually was in a form that could be merged, no
one has done that with the sdfat code :)

> sdFAT is clearly more matured and been put into more recent production softwares.
> And as I wrote in previous email, it does include some real fixes.

What fixes?  That's what I'm asking here.

How do we "know" that this is better than what we currently have today?
We don't, so it's a bit hard to tell someone, "delete the work you did
and replace it with this other random chunk of code, causing you a bunch
more work in the process for no specific reason other than it looks
'newer'." :(

> As Namjae said too, Samsung would be more interested in merging sdFAT to upstream.
> If we diverge, Samsung will have less reasons to contribute their patches to upstream.

Are they going to do this if we do take the sdfat code?  If so, great,
but I need to get someone to agree that this will happen.

> Also, I think it makes much more sense to make Samsung the maintainer of this driver
> (if they're willing to put in the manpower to do so). Asking them would be the first
> step in doing so.

They are more than willing to start contributing and being a
co-maintainer of it, it needs all the help it can get.

But "someone" from samsung, or anywhere else, has to be willing to step
up here and do this work.  Without that happening, I don't see a reason
to change at this point in time.

Rembember, kernel development happens because people do the work, which
Valdis did, and continues to do.  Throwing that away because of the
thought that someone else might do some work in the future is not how we
work.

I recommend looking at what we have in the tree now, and seeing what is
missing compared to "sdfat".  I know a lot of places in the exfat code
that needs to be fixed up, odds are they are the same stuff that needs
to be resolved in sdfat as well.

thanks,

greg k-h
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Ju Hyung Park]

On Tue, Sep 17, 2019 at 2:47 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> It's the fact that it actually was in a form that could be merged, no
> one has done that with the sdfat code :)

Well, I'm more than happy to help if you guys are happy with merging
the new base.

> What fixes?  That's what I'm asking here.

I gave this as an example in my previous email:
https://github.com/MotorolaMobilityLLC/kernel-msm/commit/7ab1657

> How do we "know" that this is better than what we currently have today?
> We don't, so it's a bit hard to tell someone, "delete the work you did
> and replace it with this other random chunk of code, causing you a bunch
> more work in the process for no specific reason other than it looks
> 'newer'." :(

The new sdFAT base I'm suggesting, is just as "random" as the one
staging tree is based on.

If exFAT gets merged to Torvald's tree, there will be a lot more eyes
interested in it.
If there's a better base, we better switch to it now and prevent
further headaches long-term.

It's really hard to compare those 2 drivers base and extract
meaningful changelogs.

But regardless, here are some diff stats:
<Full diff stat>
 Kconfig      |   79 +-
 Makefile     |   46 +-
 api.c        |  423 ----
 api.h        |  310 ---
 blkdev.c     |  409 +---
 cache.c      | 1142 ++++-----
 config.h     |   49 -
 core.c       | 5583 ++++++++++++++++++++++++--------------------
 core.h       |  196 --
 core_exfat.c | 1553 ------------
 exfat.h      | 1309 +++++++----
 exfat_fs.h   |  417 ----
 extent.c     |  351 ---
 fatent.c     |  182 --
 misc.c       |  401 ----
 nls.c        |  490 ++--
 super.c      | 5103 +++++++++++++++++++++-------------------
 upcase.c     |  740 ++++++
 upcase.h     |  407 ----
 version.h    |   29 -
 xattr.c      |  136 --
 21 files changed, 8186 insertions(+), 11169 deletions(-)

<diff-filter=M>
 Kconfig  |   79 +-
 Makefile |   46 +-
 blkdev.c |  409 +---
 cache.c  | 1142 +++++-----
 core.c   | 5583 ++++++++++++++++++++++++++----------------------
 exfat.h  | 1309 ++++++++----
 nls.c    |  490 ++---
 super.c  | 5103 ++++++++++++++++++++++---------------------
 8 files changed, 7446 insertions(+), 6715 deletions(-)

> I recommend looking at what we have in the tree now, and seeing what is
> missing compared to "sdfat".  I know a lot of places in the exfat code
> that needs to be fixed up, odds are they are the same stuff that needs
> to be resolved in sdfat as well.

Would there be any more data that I can provide?
It's really hard to go through the full diff :(

Thanks.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

RE: [PATCH] staging: exfat: add exfat filesystem code to

[Namjae Jeon]

I've summarized some of the big differences between sdfat and exfat
in linux-next.

1. sdfat has been refactored to improve compatibility, readability and
to be linux friendly.(included support mass storages larger than 2TB.)

2. sdfat has been optimized for the performance of SD-cards.
  - Support SD-card friendly block allocation and delayed allocation
    for vfat-fs only.
  - Support aligned_mpage_write for both vfat-fs and exfat-fs

3. sdfat has been optimized for the performance of general operations
    like create,lookup and readdir.

4. Fix many critical and minor bugs
 - Handle many kinds of error conditions gracefully to prevent panic.
 - Fix critical bugs related to rmdir, truncate behavior and so on...

5. Fix NLS functions

Note, that Samsung is still improving sdfat driver. For instance,
what will be realeased soon is sdfat v2.3.0, which will include support
for "UtcOffset" of "File Directory Entry", in order to satisfy
exFAT specification 7.4.

Thanks!

> -----Original Message-----
> From: Ju Hyung Park [mailto:qkrwngud825@gmail.com]
> Sent: Tuesday, September 17, 2019 3:04 PM
> To: Greg KH; namjae.jeon@samsung.com; Valdis Kletnieks
> Cc: devel@driverdev.osuosl.org; linkinjeon@gmail.com; linux-
> kernel@vger.kernel.org; alexander.levin@microsoft.com;
> sergey.senozhatsky@gmail.com; linux-fsdevel@vger.kernel.org
> Subject: Re: [PATCH] staging: exfat: add exfat filesystem code to
> 
> On Tue, Sep 17, 2019 at 2:47 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> > It's the fact that it actually was in a form that could be merged, no
> > one has done that with the sdfat code :)
> 
> Well, I'm more than happy to help if you guys are happy with merging
> the new base.
> 
> > What fixes?  That's what I'm asking here.
> 
> I gave this as an example in my previous email:
> https://github.com/MotorolaMobilityLLC/kernel-msm/commit/7ab1657
> 
> > How do we "know" that this is better than what we currently have today?
> > We don't, so it's a bit hard to tell someone, "delete the work you did
> > and replace it with this other random chunk of code, causing you a bunch
> > more work in the process for no specific reason other than it looks
> > 'newer'." :(
> 
> The new sdFAT base I'm suggesting, is just as "random" as the one
> staging tree is based on.
> 
> If exFAT gets merged to Torvald's tree, there will be a lot more eyes
> interested in it.
> If there's a better base, we better switch to it now and prevent
> further headaches long-term.
> 
> It's really hard to compare those 2 drivers base and extract
> meaningful changelogs.
> 
> But regardless, here are some diff stats:
> <Full diff stat>
>  Kconfig      |   79 +-
>  Makefile     |   46 +-
>  api.c        |  423 ----
>  api.h        |  310 ---
>  blkdev.c     |  409 +---
>  cache.c      | 1142 ++++-----
>  config.h     |   49 -
>  core.c       | 5583 ++++++++++++++++++++++++--------------------
>  core.h       |  196 --
>  core_exfat.c | 1553 ------------
>  exfat.h      | 1309 +++++++----
>  exfat_fs.h   |  417 ----
>  extent.c     |  351 ---
>  fatent.c     |  182 --
>  misc.c       |  401 ----
>  nls.c        |  490 ++--
>  super.c      | 5103 +++++++++++++++++++++-------------------
>  upcase.c     |  740 ++++++
>  upcase.h     |  407 ----
>  version.h    |   29 -
>  xattr.c      |  136 --
>  21 files changed, 8186 insertions(+), 11169 deletions(-)
> 
> <diff-filter=M>
>  Kconfig  |   79 +-
>  Makefile |   46 +-
>  blkdev.c |  409 +---
>  cache.c  | 1142 +++++-----
>  core.c   | 5583 ++++++++++++++++++++++++++----------------------
>  exfat.h  | 1309 ++++++++----
>  nls.c    |  490 ++---
>  super.c  | 5103 ++++++++++++++++++++++---------------------
>  8 files changed, 7446 insertions(+), 6715 deletions(-)
> 
> > I recommend looking at what we have in the tree now, and seeing what is
> > missing compared to "sdfat".  I know a lot of places in the exfat code
> > that needs to be fixed up, odds are they are the same stuff that needs
> > to be resolved in sdfat as well.
> 
> Would there be any more data that I can provide?
> It's really hard to go through the full diff :(
> 
> Thanks.

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

['Greg KH']

A: http://en.wikipedia.org/wiki/Top_post
Q: Were do I find info about this thing called top-posting?
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

A: No.
Q: Should I include quotations after my reply?

http://daringfireball.net/2007/07/on_top

On Wed, Sep 18, 2019 at 11:35:08AM +0900, Namjae Jeon wrote:
> I've summarized some of the big differences between sdfat and exfat
> in linux-next.
> 
> 1. sdfat has been refactored to improve compatibility, readability and
> to be linux friendly.(included support mass storages larger than 2TB.)
> 
> 2. sdfat has been optimized for the performance of SD-cards.
>   - Support SD-card friendly block allocation and delayed allocation
>     for vfat-fs only.
>   - Support aligned_mpage_write for both vfat-fs and exfat-fs
> 
> 3. sdfat has been optimized for the performance of general operations
>     like create,lookup and readdir.
> 
> 4. Fix many critical and minor bugs
>  - Handle many kinds of error conditions gracefully to prevent panic.
>  - Fix critical bugs related to rmdir, truncate behavior and so on...
> 
> 5. Fix NLS functions

Those are all wonderful things, any chances you can point out the
individual commits that reflect these bugfixes?

> Note, that Samsung is still improving sdfat driver. For instance,
> what will be realeased soon is sdfat v2.3.0, which will include support
> for "UtcOffset" of "File Directory Entry", in order to satisfy
> exFAT specification 7.4.

If Samsung is going to continue to keep its internal tree for this
driver, there's no way we can take a code dump today and expect things
to keep in sync.

If Samsung agrees to do development of this codebase upstream in the
kernel tree, I will be glad to consider moving to this codebase instead
and working together on it.  Otherwise it makes no sense as things
instantly diverge with no way of keeping in sync (we only can commit to
one tree, while you can in both.)

If Samsung wishes to use their sdfat codebase as the "seed" to work from
for this, please submit a patch adding the latest version to the kernel
tree and we can compare and work from there.

thanks,

greg k-h
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Sergey Senozhatsky]

On (09/18/19 08:16), 'Greg KH' wrote:
[..]
> > Note, that Samsung is still improving sdfat driver. For instance,
> > what will be realeased soon is sdfat v2.3.0, which will include support
> > for "UtcOffset" of "File Directory Entry", in order to satisfy
> > exFAT specification 7.4.
>
[..]
> If Samsung wishes to use their sdfat codebase as the "seed" to work from
> for this, please submit a patch adding the latest version to the kernel
> tree and we can compare and work from there.

Isn't it what Ju Hyung did? He took sdfat codebase (the most recent
among publicly available) as the seed, cleaned it up a bit and submitted
as a patch. Well, technically, Valdis did the same, it's just he forked
a slightly more outdated (and not anymore used by Samsung) codebase.

	-ss
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

['Greg KH']

On Wed, Sep 18, 2019 at 03:33:04PM +0900, Sergey Senozhatsky wrote:
> On (09/18/19 08:16), 'Greg KH' wrote:
> [..]
> > > Note, that Samsung is still improving sdfat driver. For instance,
> > > what will be realeased soon is sdfat v2.3.0, which will include support
> > > for "UtcOffset" of "File Directory Entry", in order to satisfy
> > > exFAT specification 7.4.
> >
> [..]
> > If Samsung wishes to use their sdfat codebase as the "seed" to work from
> > for this, please submit a patch adding the latest version to the kernel
> > tree and we can compare and work from there.
> 
> Isn't it what Ju Hyung did? He took sdfat codebase (the most recent
> among publicly available) as the seed, cleaned it up a bit and submitted
> as a patch.

He did?  I do not see a patch anywhere, what is the message-id of it?

> Well, technically, Valdis did the same, it's just he forked a slightly
> more outdated (and not anymore used by Samsung) codebase.

He took the "best known at the time" codebase, as we had nothing else to
work with.

thanks,

greg k-h
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Sergey Senozhatsky]

On (09/18/19 10:26), 'Greg KH' wrote:
> On Wed, Sep 18, 2019 at 03:33:04PM +0900, Sergey Senozhatsky wrote:
> > On (09/18/19 08:16), 'Greg KH' wrote:
> > [..]
> > > > Note, that Samsung is still improving sdfat driver. For instance,
> > > > what will be realeased soon is sdfat v2.3.0, which will include support
> > > > for "UtcOffset" of "File Directory Entry", in order to satisfy
> > > > exFAT specification 7.4.
> > >
> > [..]
> > > If Samsung wishes to use their sdfat codebase as the "seed" to work from
> > > for this, please submit a patch adding the latest version to the kernel
> > > tree and we can compare and work from there.
> > 
> > Isn't it what Ju Hyung did? He took sdfat codebase (the most recent
> > among publicly available) as the seed, cleaned it up a bit and submitted
> > as a patch.
> 
> He did?  I do not see a patch anywhere, what is the message-id of it?

Sorry. No, he did not. I somehow thought that he did, but it seems that
I just looked at his github and emails.

> > Well, technically, Valdis did the same, it's just he forked a slightly
> > more outdated (and not anymore used by Samsung) codebase.
> 
> He took the "best known at the time" codebase, as we had nothing else to
> work with.

Well, then Valdis probably took it a long long time ago. Current
"best known" is v2.2, publicly available under GPLv2 at opensource.samsung.com

	-ss
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Ju Hyung Park]

On Wed, Sep 18, 2019 at 5:33 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> He did?  I do not see a patch anywhere, what is the message-id of it?

I'm just repeating myself at this point, but again, I'm more than
willing to work on a patch.
I just want to make it clear on how should I.

> He took the "best known at the time" codebase, as we had nothing else to
> work with.

It wasn't the "best known at the time". sdFAT has been around now for years.

Thanks.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Dan Carpenter]

On Wed, Sep 18, 2019 at 06:01:09PM +0900, Ju Hyung Park wrote:
> On Wed, Sep 18, 2019 at 5:33 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> > He did?  I do not see a patch anywhere, what is the message-id of it?
> 
> I'm just repeating myself at this point, but again, I'm more than
> willing to work on a patch.
> I just want to make it clear on how should I.

Put it in drivers/staging/sdfat/.

But really we want someone from Samsung to say that they will treat
the staging version as upstream.  It doesn't work when people apply
fixes to their version and a year later back port the fixes into
staging.  The staging tree is going to have tons and tons of white space
fixes so backports are a pain.  All development needs to be upstream
first where the staging driver is upstream.  Otherwise we should just
wait for Samsung to get it read to be merged in fs/ and not through the
staging tree.

regards,
dan carpenter


_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Ju Hyung Park]

Hi Dan,

On Wed, Sep 18, 2019 at 6:27 PM Dan Carpenter <dan.carpenter@oracle.com> wrote:
> Put it in drivers/staging/sdfat/.

It'll conflict with the current exfat staging drivers.
And moreover, I don't think it makes sense to use sdfat naming in mainline.

Samsung uses it since it handles all fat filesystems.
From what I can tell, that's not in mainline's interests:
https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git/commit/?h=staging-next&id=58985a9d2d03e977db93bf574a16162766a318fe

What I'm proposing is to remove the current exfat drivers and add
sdfat-based one(that I removed fat16/32 handlings and renamed to
exfat).

> But really we want someone from Samsung to say that they will treat
> the staging version as upstream.

Agreed.
Perhaps Namjae didn't pick up our questions with all those mails we
sent during last few days.

Maybe ping him again?

Thanks.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Dan Carpenter]

On Wed, Sep 18, 2019 at 06:53:49PM +0900, Ju Hyung Park wrote:
> Hi Dan,
> 
> On Wed, Sep 18, 2019 at 6:27 PM Dan Carpenter <dan.carpenter@oracle.com> wrote:
> > Put it in drivers/staging/sdfat/.
> 
> It'll conflict with the current exfat staging drivers.

Use Kconfig.

> And moreover, I don't think it makes sense to use sdfat naming in mainline.

The directory doesn't need to be permanent.

regards,
dan carpenter

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Ju Hyung Park]

On Wed, Sep 18, 2019 at 7:09 PM Dan Carpenter <dan.carpenter@oracle.com> wrote:
> Use Kconfig.

Not just that.
There are a lot of non-static functions that's not marked ex/sdfat-specific.
(which we would have to clean it up eventually)

Even with sdFAT base, there are some non-static functions named as exfat.

Figuring out a solution for this is pretty pointless imho when one of
the drivers will be dropped soon(ish) anyways.

Thanks.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Greg KH]

On Wed, Sep 18, 2019 at 07:46:25PM +0900, Ju Hyung Park wrote:
> On Wed, Sep 18, 2019 at 7:09 PM Dan Carpenter <dan.carpenter@oracle.com> wrote:
> > Use Kconfig.
> 
> Not just that.
> There are a lot of non-static functions that's not marked ex/sdfat-specific.
> (which we would have to clean it up eventually)

Then clean them up :)

> Even with sdFAT base, there are some non-static functions named as exfat.

Then just force both filesystems to only be built as a module and all
should be fine, right?

> Figuring out a solution for this is pretty pointless imho when one of
> the drivers will be dropped soon(ish) anyways.

Given we only have one filesytem that is submitted in patch form, I
think people are making a lot of noise over nothing :)

thanks,

greg k-h
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

RE: [PATCH] staging: exfat: add exfat filesystem code to

[Namjae Jeon]

[..]
> Put it in drivers/staging/sdfat/.
> 
> But really we want someone from Samsung to say that they will treat
> the staging version as upstream.  It doesn't work when people apply
> fixes to their version and a year later back port the fixes into
> staging.  The staging tree is going to have tons and tons of white space
> fixes so backports are a pain.  All development needs to be upstream
> first where the staging driver is upstream.  Otherwise we should just
> wait for Samsung to get it read to be merged in fs/ and not through the
> staging tree.
Quite frankly,
This whole thing came as a huge-huge surprise to us, we did not initiate
upstreaming of exfat/sdfat code and, as of this moment, I'm not exactly
sure that we are prepared for any immediate radical changes to our internal
development process which people all of a sudden want from us. I need to
discuss with related people on internal thread.
please wait a while:)

Thanks!
> 
> regards,
> dan carpenter
> 


_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

RE: [PATCH] staging: exfat: add exfat filesystem code to

[Namjae Jeon]

> [..]
> > Put it in drivers/staging/sdfat/.
> >
> > But really we want someone from Samsung to say that they will treat
> > the staging version as upstream.  It doesn't work when people apply
> > fixes to their version and a year later back port the fixes into
> > staging.  The staging tree is going to have tons and tons of white space
> > fixes so backports are a pain.  All development needs to be upstream
> > first where the staging driver is upstream.  Otherwise we should just
> > wait for Samsung to get it read to be merged in fs/ and not through the
> > staging tree.
> Quite frankly,
> This whole thing came as a huge-huge surprise to us, we did not initiate
> upstreaming of exfat/sdfat code and, as of this moment, I'm not exactly
> sure that we are prepared for any immediate radical changes to our internal
> development process which people all of a sudden want from us. I need to
> discuss with related people on internal thread.
> please wait a while:)
We decide to contribute sdfat directly and treat upstream exfat.
Perhaps more time is needed for patch preparation(exfat rename + vfat removal
+ clean-up) and internal processes. After contributing sdfat v2.2.0 as the base,
We will also provide change-set of sdfat v2.3.0 later.

Thanks!
> 
> Thanks!
> >
> > regards,
> > dan carpenter
> >


_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

['Greg KH']

On Mon, Sep 30, 2019 at 01:25:13PM +0900, Namjae Jeon wrote:
> 
> > [..]
> > > Put it in drivers/staging/sdfat/.
> > >
> > > But really we want someone from Samsung to say that they will treat
> > > the staging version as upstream.  It doesn't work when people apply
> > > fixes to their version and a year later back port the fixes into
> > > staging.  The staging tree is going to have tons and tons of white space
> > > fixes so backports are a pain.  All development needs to be upstream
> > > first where the staging driver is upstream.  Otherwise we should just
> > > wait for Samsung to get it read to be merged in fs/ and not through the
> > > staging tree.
> > Quite frankly,
> > This whole thing came as a huge-huge surprise to us, we did not initiate
> > upstreaming of exfat/sdfat code and, as of this moment, I'm not exactly
> > sure that we are prepared for any immediate radical changes to our internal
> > development process which people all of a sudden want from us. I need to
> > discuss with related people on internal thread.
> > please wait a while:)
> We decide to contribute sdfat directly and treat upstream exfat.
> Perhaps more time is needed for patch preparation(exfat rename + vfat removal
> + clean-up) and internal processes. After contributing sdfat v2.2.0 as the base,
> We will also provide change-set of sdfat v2.3.0 later.

That's wonderful to hear!  If you need help getting patches into
mergable shape, just let us know.

thanks,

greg k-h
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Ju Hyung Park]

Hi Greg,

On Sun, Sep 15, 2019 at 10:54 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> Note, this just showed up publically on August 12, where were you with
> all of this new code before then?  :)

My sdFAT port, exfat-nofuse and the one on the staging tree, were all
made by Samsung.
And unless you guys had a chance to talk to Samsung developers
directly, those all share the same faith of lacking proper development
history.

The source I used was from http://opensource.samsung.com, which
provides kernel sources as tar.gz files.
There is no code history available.

> For the in-kernel code, we would have to rip out all of the work you did
> for all older kernels, so that's a non-starter right there.

I'm aware.
I'm just letting mainline know that there is potentially another (much
better) base that could be upstreamed.

If you want me to rip out older kernel support for upstreaming, I'm
more than happy to do so.

> As for what codebase to work off of, I don't want to say it is too late,
> but really, this shows up from nowhere and we had to pick something so
> we found the best we could at that point in time.

To be honest, whole public exFAT sources are all from nowhere unless
you had internal access to Samsung's development archive.
The one in the current staging tree isn't any better.

I'm not even sure where the staging driver is from, actually.

Samsung used the 1.2.x versioning until they switched to a new code
base - sdFAT.
The one in the staging tree is marked version 1.3.0(exfat_super.c).
I failed to find anything 1.3.x from Samsung's public kernel sources.

The last time exFAT 1.2.x was used was in Galaxy S7(released in 2016).
Mine was originally based on sdFAT 2.1.10, used in Galaxy S10(released
in March 2019) and it just got updated to 2.2.0, used in Galaxy
Note10(released in August 2019).

> Is there anything specific in the codebase you have now, that is lacking
> in the in-kernel code?  Old-kernel-support doesn't count here, as we
> don't care about that as it is not applicable.  But functionality does
> matter, what has been added here that we can make use of?

This is more of a suggestion of
"Let's base on a *much more recent* snapshot for the community to work on",
since the current one on the staging tree also lacks development history.

The diff is way too big to even start understanding the difference.


With that said though, I do have some vague but real reason as to why
sdFAT base is better.

With some major Android vendors showing interests in supporting exFAT,
Motorola notably published their work on public Git repository with
full development history(the only vendor to do this that I'm aware
of).
Commits like this:
https://github.com/MotorolaMobilityLLC/kernel-msm/commit/7ab1657 is
not merged to exFAT(including the current staging tree one) while it
did for sdFAT.


The only thing I regret is not working on porting sdFAT sooner.
I definitely didn't anticipate Microsoft to suddenly lift legal issues
on upstreaming exFAT just around when I happen to gain interest in
porting sdFAT.

If my port happened sooner, it would have been a no-brainer for it to
be considered as a top candidate for upstreaming.

> And do you have any "real" development history to look at instead of the
> "one giant commit" of the initial code drop?  That is where we could
> actually learn what has changed over time.  Your repo as-is shows none
> of the interesting bits :(

As I mentioned, development history is unobtainable, even for the
current staging tree or exfat-nofuse.
(If you guys took exfat-nofuse, you can also see that there's barely
any real exFAT-related development done in that tree. Everything is
basically fixes for newer kernel versions.)

The best I could do, if someone's interested, is to diff all versions
of exFAT/sdFAT throughout the Samsung's kernel versions, but that
still won't give us reasons as to why the changes were made.

TL;DR
My suggestion - Let's base on a much newer driver that's matured more,
contains more fixes, gives (slightly?) better performance and
hopefully has better code quality.

Both drivers are horrible.
You said it yourself(for the current staging one), and even for my new
sdFAT-base proposal, I'm definitely not comfortable seeing this kind
of crap in mainline:
https://github.com/arter97/exfat-linux/commit/0f1ddde

However, it's clear to me that the sdFAT base is less-horrible.

Please let me know what you think.

> thanks,
>
> greg kh

Thanks.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Greg KH]

On Sat, Sep 14, 2019 at 10:39:51PM +0900, Park Ju Hyung wrote:
> Hi.
> 
> I just noticed that this exfat-staging drivers are based on the old 
> Samsung's 1.x exFAT drivers.
> 
> I've been working to get the newer Samsung's driver(now named "sdFAT") 
> to fit better for general Linux users, and I believe it can provide a 
> better base for the community to work on(and hopefully complies better 
> to the mainline coding standard).
> 
> GitHub link
> https://github.com/arter97/exfat-linux
> 
> I also included some rudimentary benchmark results.
> 
> I encourage mainline developers to explore this driver base and see if 
> it's worth to switch, since it's the early days of exfat-staging.

Note, this just showed up publically on August 12, where were you with
all of this new code before then?  :)

> To others watching this thread:
> It's more than likely that you can start using exFAT reliably right 
> away by following the link above. It's tested on all major LTS kernels 
> ranging from 3.4 to 4.19 and the ones Canonical uses for Ubuntu: 3.4, 
> 3.10, 3.18, 4.1, 4.4, 4.9, 4.14, 4.19 and 4.15, 5.0, 5.2, and 5.3-rc.

For the in-kernel code, we would have to rip out all of the work you did
for all older kernels, so that's a non-starter right there.

As for what codebase to work off of, I don't want to say it is too late,
but really, this shows up from nowhere and we had to pick something so
we found the best we could at that point in time.

Is there anything specific in the codebase you have now, that is lacking
in the in-kernel code?  Old-kernel-support doesn't count here, as we
don't care about that as it is not applicable.  But functionality does
matter, what has been added here that we can make use of?

And do you have any "real" development history to look at instead of the
"one giant commit" of the initial code drop?  That is where we could
actually learn what has changed over time.  Your repo as-is shows none
of the interesting bits :(

thanks,

greg kh
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH] staging: exfat: add exfat filesystem code to

[Park Ju Hyung]

Hi.

I just noticed that this exfat-staging drivers are based on the old 
Samsung's 1.x exFAT drivers.

I've been working to get the newer Samsung's driver(now named "sdFAT") 
to fit better for general Linux users, and I believe it can provide a 
better base for the community to work on(and hopefully complies better 
to the mainline coding standard).

GitHub link
https://github.com/arter97/exfat-linux

I also included some rudimentary benchmark results.

I encourage mainline developers to explore this driver base and see if 
it's worth to switch, since it's the early days of exfat-staging.

To others watching this thread:
It's more than likely that you can start using exFAT reliably right 
away by following the link above. It's tested on all major LTS kernels 
ranging from 3.4 to 4.19 and the ones Canonical uses for Ubuntu: 3.4, 
3.10, 3.18, 4.1, 4.4, 4.9, 4.14, 4.19 and 4.15, 5.0, 5.2, and 5.3-rc.

Thanks.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel