Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

[Tom Seewald]

Hi,

Has this been abandoned or will there be future attempts at syncing the
in-kernel zstd with the upstream project?

Tom

Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

[Neal Gompa]

On Tue, Sep 28, 2021 at 8:24 PM Tom Seewald <tseewald@gmail.com> wrote:
>
> Hi,
>
> Has this been abandoned or will there be future attempts at syncing the
> in-kernel zstd with the upstream project?
>

With the kind of crap that Nick has been getting from everyone, I'd be
surprised if he takes it up again anytime soon. As a bystander, I've been
incredibly disappointed with how this has "progressed" (read: stalled)
even though Nick is the developer and maintainer of zstd code.




--
真実はいつも一つ！/ Always, there's only one truth!

Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

[Nick Terrell]

> On Sep 28, 2021, at 5:22 PM, Tom Seewald <tseewald@gmail.com> wrote:
> 
> Hi,
> 
> Has this been abandoned or will there be future attempts at syncing the
> in-kernel zstd with the upstream project?
> 
> Tom

Sorry for the lack of action, but this has not been abandoned. I’ve just been
preparing a rebased patch-set last week, so expect to see some action soon.
Since we’re not in a merge window, I’m unsure if it is best to send out the
updated patches now, or wait until the merge window is open, but I’m about to
pose that question to the LKML.

This work has been on my back burner, because I’ve been busy with work on
Zstd and other projects, and have had a hard time justifying to myself spending
too much time on this, since progress has been so slow.

Best,
Nick Terrell

Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

[David Sterba]

On Wed, Sep 29, 2021 at 01:30:26AM +0000, Nick Terrell wrote:
> > On Sep 28, 2021, at 5:22 PM, Tom Seewald <tseewald@gmail.com> wrote:
> > Has this been abandoned or will there be future attempts at syncing the
> > in-kernel zstd with the upstream project?
> 
> Sorry for the lack of action, but this has not been abandoned. I’ve just been
> preparing a rebased patch-set last week, so expect to see some action soon.
> Since we’re not in a merge window, I’m unsure if it is best to send out the
> updated patches now, or wait until the merge window is open, but I’m about to
> pose that question to the LKML.

If you send it once merge window is open it's unlikely to be merged. The
code must be ready before it opens and part of linux-next for a week at
least if not more.

> This work has been on my back burner, because I’ve been busy with work on
> Zstd and other projects, and have had a hard time justifying to myself spending
> too much time on this, since progress has been so slow.

What needs to be done from my POV:

- refresh the patches on top of current mainline, eg. v5.15-rc3

- make sure it compiles and works with current in-kernel users of zstd,
  ie. with btrfs in particular, I can do some tests too

- push the patches to a public branch eg. on k.org or github

- ask for adding the branch to linux-next

- try to get some feedback from people that were objecting in the past,
  and of course gather acks or supportive feedback

- once merge window opens, send a pull request to Linus, write the
  rationale why we want this change and summarize the evolution of the
  patchset and why the full version update is perhaps the way forward

RE: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

[Paul Jones]

> -----Original Message-----
> From: David Sterba <dsterba@suse.cz>
> Sent: Wednesday, 29 September 2021 8:07 PM
> To: Nick Terrell <terrelln@fb.com>
> Cc: B093B859-53CC-4818-8CC3-A317F4872AD6@fb.com; linux-
> btrfs@vger.kernel.org; linux-kernel@vger.kernel.org
> Subject: Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10
> 
> On Wed, Sep 29, 2021 at 01:30:26AM +0000, Nick Terrell wrote:
> > > On Sep 28, 2021, at 5:22 PM, Tom Seewald <tseewald@gmail.com>
> wrote:
> > > Has this been abandoned or will there be future attempts at syncing
> > > the in-kernel zstd with the upstream project?
> >
> > Sorry for the lack of action, but this has not been abandoned. I’ve
> > just been preparing a rebased patch-set last week, so expect to see some
> action soon.
> > Since we’re not in a merge window, I’m unsure if it is best to send
> > out the updated patches now, or wait until the merge window is open,
> > but I’m about to pose that question to the LKML.
> 
> If you send it once merge window is open it's unlikely to be merged. The
> code must be ready before it opens and part of linux-next for a week at least
> if not more.
> 
> > This work has been on my back burner, because I’ve been busy with work
> > on Zstd and other projects, and have had a hard time justifying to
> > myself spending too much time on this, since progress has been so slow.
> 
> What needs to be done from my POV:
> 
> - refresh the patches on top of current mainline, eg. v5.15-rc3
> 
> - make sure it compiles and works with current in-kernel users of zstd,
>   ie. with btrfs in particular, I can do some tests too

I have been running this patchset with btrfs on the latest stable kernels (5.12-14), and have not encountered any issues at all.
For AMD64 and AARCH64 you can add my
Tested-by: Paul Jones <paul@pauljones.id.au>

 
> - push the patches to a public branch eg. on k.org or github
> 
> - ask for adding the branch to linux-next
> 
> - try to get some feedback from people that were objecting in the past,
>   and of course gather acks or supportive feedback
> 
> - once merge window opens, send a pull request to Linus, write the
>   rationale why we want this change and summarize the evolution of the
>   patchset and why the full version update is perhaps the way forward

Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

[Nick Terrell]

> On Sep 29, 2021, at 3:06 AM, David Sterba <dsterba@suse.cz> wrote:
> 
> On Wed, Sep 29, 2021 at 01:30:26AM +0000, Nick Terrell wrote:
>>> On Sep 28, 2021, at 5:22 PM, Tom Seewald <tseewald@gmail.com> wrote:
>>> Has this been abandoned or will there be future attempts at syncing the
>>> in-kernel zstd with the upstream project?
>> 
>> Sorry for the lack of action, but this has not been abandoned. I’ve just been
>> preparing a rebased patch-set last week, so expect to see some action soon.
>> Since we’re not in a merge window, I’m unsure if it is best to send out the
>> updated patches now, or wait until the merge window is open, but I’m about to
>> pose that question to the LKML.
> 
> If you send it once merge window is open it's unlikely to be merged. The
> code must be ready before it opens and part of linux-next for a week at
> least if not more.
> 
>> This work has been on my back burner, because I’ve been busy with work on
>> Zstd and other projects, and have had a hard time justifying to myself spending
>> too much time on this, since progress has been so slow.
> 
> What needs to be done from my POV:
> 
> - refresh the patches on top of current mainline, eg. v5.15-rc3
> 
> - make sure it compiles and works with current in-kernel users of zstd,
>  ie. with btrfs in particular, I can do some tests too
> 
> - push the patches to a public branch eg. on k.org or github
> 
> - ask for adding the branch to linux-next
> 
> - try to get some feedback from people that were objecting in the past,
>  and of course gather acks or supportive feedback
> 
> - once merge window opens, send a pull request to Linus, write the
>  rationale why we want this change and summarize the evolution of the
>  patchset and why the full version update is perhaps the way forward

Thanks for the clear explanation, that was very helpful! I’ve already rebased
and re-tested myself. I’ll send out a request asking it to be added to linux-next,
and try to gather feedback/acks on that thread. I’ll prepare the rationale for the
linux-next request, so you all can get a chance to read it.

Thanks,
Nick

Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

[Oleksandr Natalenko]

Hello.

On středa 29. září 2021 3:30:26 CEST Nick Terrell wrote:
> Sorry for the lack of action, but this has not been abandoned. I’ve just
> been
> preparing a rebased patch-set last week, so expect to see some action
> soon. Since we’re not in a merge window, I’m unsure if it is best to send
> out the updated patches now, or wait until the merge window is open, but
> I’m about to pose that question to the LKML.
> 
> This work has been on my back burner, because I’ve been busy with work on
> Zstd and other projects, and have had a hard time justifying to myself
> spending
> too much time on this, since progress has been so slow.

Mind Cc'ing me on your new submission again please? I'm still running your old 
one with 5.14 and 5.15, and it works flawlessly for me.

Thanks.

-- 
Oleksandr Natalenko (post-factum)

Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

[Jean-Denis Girard]

Hi list,

Le 29/04/2021 à 15:31, Nick Terrell a écrit :
> From: Nick Terrell <terrelln@fb.com>
> 
> Please pull from
> 
>    git@github.com:terrelln/linux.git tags/v11-zstd-1.4.10
> 
> to get these changes. Alternatively the patchset is included.
> 
> This patchset lists me as the maintainer for zstd and upgrades the zstd library
> to the latest upstream release. The current zstd version in the kernel is a
> modified version of upstream zstd-1.3.1. At the time it was integrated, zstd
> wasn't ready to be used in the kernel as-is. But, it is now possible to use
> upstream zstd directly in the kernel.
> 
> I have not yet released zstd-1.4.10 upstream. I want the zstd version in the
> kernel to match up with a known upstream release, so we know exactly what code
> is running. Whenever this patchset is ready for merge, I will cut a release at
> the upstream commit that gets merged. This should not be necessary for future
> releases.
> 
> The kernel zstd library is automatically generated from upstream zstd. A script
> makes the necessary changes and imports it into the kernel. The changes are:
> 
> 1. Replace all libc dependencies with kernel replacements and rewrite includes.
> 2. Remove unncessary portability macros like: #if defined(_MSC_VER).
> 3. Use the kernel xxhash instead of bundling it.
> 
> This automation gets tested every commit by upstream's continuous integration.
> When we cut a new zstd release, we will submit a patch to the kernel to update
> the zstd version in the kernel.
> 
> I've updated zstd to upstream with one big patch because every commit must build,
> so that precludes partial updates. Since the commit is 100% generated, I hope the
> review burden is lightened. I considered replaying upstream commits, but that is
> not possible because there have been ~3500 upstream commits since the last zstd
> import, and the commits don't all build individually. The bulk update preserves
> bisectablity because bugs can be bisected to the zstd version update. At that
> point the update can be reverted, and we can work with upstream to find and fix
> the bug. After this big switch in how the kernel consumes zstd, future patches
> will be smaller, because they will only have one upstream release worth of
> changes each.
> 
> This patchset adds a new kernel-style wrapper around zstd. This wrapper API is
> functionally equivalent to the subset of the current zstd API that is currently
> used. The wrapper API changes to be kernel style so that the symbols don't
> collide with zstd's symbols. The update to zstd-1.4.6 maintains the same API
> and preserves the semantics, so that none of the callers need to be updated.
> 
> This patchset comes in 2 parts:
> 1. The first 2 patches prepare for the zstd upgrade. The first patch adds the
>     new kernel style API so zstd can be upgraded without modifying any callers.
>     The second patch adds an indirection for the lib/decompress_unzstd.c
>     including of all decompression source files.
> 2. Import zstd-1.4.10. This patch is completely generated from upstream using
>     automated tooling.
> 
> I tested every caller of zstd on x86_64. I tested both after the 1.4.10 upgrade
> using the compatibility wrapper, and after the final patch in this series.
> 
> I tested kernel and initramfs decompression in i386 and arm.
> 
> I ran benchmarks to compare the current zstd in the kernel with zstd-1.4.6.
> I benchmarked on x86_64 using QEMU with KVM enabled on an Intel i9-9900k.
> I found:
> * BtrFS zstd compression at levels 1 and 3 is 5% faster
> * BtrFS zstd decompression+read is 15% faster
> * SquashFS zstd decompression+read is 15% faster
> * F2FS zstd compression+write at level 3 is 8% faster
> * F2FS zstd decompression+read is 20% faster
> * ZRAM decompression+read is 30% faster
> * Kernel zstd decompression is 35% faster
> * Initramfs zstd decompression+build is 5% faster
> 
> The latest zstd also offers bug fixes. For example the problem with large kernel
> decompression has been fixed upstream for over 2 years
> https://lkml.org/lkml/2020/9/29/27.
> 
> Please let me know if there is anything that I can do to ease the way for these
> patches. I think it is important because it gets large performance improvements,
> contains bug fixes, and is switching to a more maintainable model of consuming
> upstream zstd directly, making it easy to keep up to date.
> 
> Best,
> Nick Terrell


I've been using this series on stable kernel since 5.12.3 (now on 
5.12.10) on my main system without issues.

Tested-by: Jean-Denis Girard <jd.girard@sysnux.pf>


Thanks,
-- 
Jean-Denis Girard

SysNux                   Systèmes   Linux   en   Polynésie  française
https://www.sysnux.pf/   Tél: +689 40.50.10.40 / GSM: +689 87.797.527

Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

[Jean-Denis Girard]

Hi list,

Le 29/04/2021 à 15:31, Nick Terrell a écrit :
> From: Nick Terrell <terrelln@fb.com>
> 
> Please pull from
> 
>    git@github.com:terrelln/linux.git tags/v11-zstd-1.4.10
> 
> to get these changes. Alternatively the patchset is included.
> 
> This patchset lists me as the maintainer for zstd and upgrades the zstd library
> to the latest upstream release. The current zstd version in the kernel is a
> modified version of upstream zstd-1.3.1. At the time it was integrated, zstd
> wasn't ready to be used in the kernel as-is. But, it is now possible to use
> upstream zstd directly in the kernel.
> 
> I have not yet released zstd-1.4.10 upstream. I want the zstd version in the
> kernel to match up with a known upstream release, so we know exactly what code
> is running. Whenever this patchset is ready for merge, I will cut a release at
> the upstream commit that gets merged. This should not be necessary for future
> releases.
> 
> The kernel zstd library is automatically generated from upstream zstd. A script
> makes the necessary changes and imports it into the kernel. The changes are:
> 
> 1. Replace all libc dependencies with kernel replacements and rewrite includes.
> 2. Remove unncessary portability macros like: #if defined(_MSC_VER).
> 3. Use the kernel xxhash instead of bundling it.
> 
> This automation gets tested every commit by upstream's continuous integration.
> When we cut a new zstd release, we will submit a patch to the kernel to update
> the zstd version in the kernel.
> 
> I've updated zstd to upstream with one big patch because every commit must build,
> so that precludes partial updates. Since the commit is 100% generated, I hope the
> review burden is lightened. I considered replaying upstream commits, but that is
> not possible because there have been ~3500 upstream commits since the last zstd
> import, and the commits don't all build individually. The bulk update preserves
> bisectablity because bugs can be bisected to the zstd version update. At that
> point the update can be reverted, and we can work with upstream to find and fix
> the bug. After this big switch in how the kernel consumes zstd, future patches
> will be smaller, because they will only have one upstream release worth of
> changes each.
> 
> This patchset adds a new kernel-style wrapper around zstd. This wrapper API is
> functionally equivalent to the subset of the current zstd API that is currently
> used. The wrapper API changes to be kernel style so that the symbols don't
> collide with zstd's symbols. The update to zstd-1.4.6 maintains the same API
> and preserves the semantics, so that none of the callers need to be updated.
> 
> This patchset comes in 2 parts:
> 1. The first 2 patches prepare for the zstd upgrade. The first patch adds the
>     new kernel style API so zstd can be upgraded without modifying any callers.
>     The second patch adds an indirection for the lib/decompress_unzstd.c
>     including of all decompression source files.
> 2. Import zstd-1.4.10. This patch is completely generated from upstream using
>     automated tooling.
> 
> I tested every caller of zstd on x86_64. I tested both after the 1.4.10 upgrade
> using the compatibility wrapper, and after the final patch in this series.
> 
> I tested kernel and initramfs decompression in i386 and arm.
> 
> I ran benchmarks to compare the current zstd in the kernel with zstd-1.4.6.
> I benchmarked on x86_64 using QEMU with KVM enabled on an Intel i9-9900k.
> I found:
> * BtrFS zstd compression at levels 1 and 3 is 5% faster
> * BtrFS zstd decompression+read is 15% faster
> * SquashFS zstd decompression+read is 15% faster
> * F2FS zstd compression+write at level 3 is 8% faster
> * F2FS zstd decompression+read is 20% faster
> * ZRAM decompression+read is 30% faster
> * Kernel zstd decompression is 35% faster
> * Initramfs zstd decompression+build is 5% faster
> 
> The latest zstd also offers bug fixes. For example the problem with large kernel
> decompression has been fixed upstream for over 2 years
> https://lkml.org/lkml/2020/9/29/27.
> 
> Please let me know if there is anything that I can do to ease the way for these
> patches. I think it is important because it gets large performance improvements,
> contains bug fixes, and is switching to a more maintainable model of consuming
> upstream zstd directly, making it easy to keep up to date.
> 
> Best,
> Nick Terrell


I've been using this series on stable kernel since 5.12.3 (now on 
5.12.10) on my main system without issues.

Tested-by: Jean-Denis Girard <jd.girard@sysnux.pf>


Thanks,
-- 
Jean-Denis Girard

SysNux                   Systèmes   Linux   en   Polynésie  française
https://www.sysnux.pf/   Tél: +689 40.50.10.40 / GSM: +689 87.797.527

Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

[David Sterba]

On Tue, May 11, 2021 at 08:53:41PM +0000, Nick Terrell wrote:
> Pinging this series. Is there anything I should do to help get this
> merged?
> 
> The use of zstd in the kernel is continuously increasing over time,
> both in terms of number of use cases, and number of users that
> actually enable zstd compression in production. E.g. Fedora is
> making btrfs with zstd compression enabled the default.
> 
> I would love to see the zstd code updated to the latest upstream
> and be kept up to date. The latest upstream brings bug fixes, and
> significant performance improvements. Additionally, the latest
> upstream code is continuously fuzzed.

The btrfs community and I in particular have interest to get zstd
updated but also there's the patch 3 that goes against what kernel
requires regarding patch size and logical split of changes.

That the update is so large shouldn't have happened, it covers 3 years
of development, the syncs should have happened more often, but here we
are.

Other points that have been raised in the past:

* new wrappers - there are new wrappers changing users of the API, the
  new names are more conforming, eg ZSTD_decompressDCtx -> zstd_decompress_dctx,
  sounds like an improvement to me

* high stack usage - mentioned in patch 3, slight increase but bounded
  and upstream now monitors that so it does not increase

Other points that are worth mentioning:

* bisectability - the version switch happens in one patch, so the
  effects before/after the patch are only runtime as there's no change
  in format etc, so ok

* will be maintained - no such huge update should happen again

So I suggest to merge in current form. I'm not sure what was the
original plan if it was supposed to go via Herbert's crypto tree, but
that was before Nick added himself as maintainer.

I think that Nick can send the pull request to Linus, perhaps with acks
to all changes that are in the non-zstd code (patch 1).

Cover letter v11: https://lore.kernel.org/linux-btrfs/20210430013157.747152-1-nickrterrell@gmail.com/

Re: [GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

[Nick Terrell]

> On Apr 29, 2021, at 6:31 PM, Nick Terrell <nickrterrell@gmail.com> wrote:
> 
> From: Nick Terrell <terrelln@fb.com>
> 
> Please pull from
> 
>  git@github.com:terrelln/linux.git tags/v11-zstd-1.4.10
> 
> to get these changes. Alternatively the patchset is included.

Hi all,

Pinging this series. Is there anything I should do to help get this
merged?

The use of zstd in the kernel is continuously increasing over time,
both in terms of number of use cases, and number of users that
actually enable zstd compression in production. E.g. Fedora is
making btrfs with zstd compression enabled the default.

I would love to see the zstd code updated to the latest upstream
and be kept up to date. The latest upstream brings bug fixes, and
significant performance improvements. Additionally, the latest
upstream code is continuously fuzzed.

Thanks,
Nick

> This patchset lists me as the maintainer for zstd and upgrades the zstd library
> to the latest upstream release. The current zstd version in the kernel is a
> modified version of upstream zstd-1.3.1. At the time it was integrated, zstd
> wasn't ready to be used in the kernel as-is. But, it is now possible to use
> upstream zstd directly in the kernel.
> 
> I have not yet released zstd-1.4.10 upstream. I want the zstd version in the
> kernel to match up with a known upstream release, so we know exactly what code
> is running. Whenever this patchset is ready for merge, I will cut a release at
> the upstream commit that gets merged. This should not be necessary for future
> releases.
> 
> The kernel zstd library is automatically generated from upstream zstd. A script
> makes the necessary changes and imports it into the kernel. The changes are:
> 
> 1. Replace all libc dependencies with kernel replacements and rewrite includes.
> 2. Remove unncessary portability macros like: #if defined(_MSC_VER).
> 3. Use the kernel xxhash instead of bundling it.
> 
> This automation gets tested every commit by upstream's continuous integration.
> When we cut a new zstd release, we will submit a patch to the kernel to update
> the zstd version in the kernel.
> 
> I've updated zstd to upstream with one big patch because every commit must build,
> so that precludes partial updates. Since the commit is 100% generated, I hope the
> review burden is lightened. I considered replaying upstream commits, but that is
> not possible because there have been ~3500 upstream commits since the last zstd
> import, and the commits don't all build individually. The bulk update preserves
> bisectablity because bugs can be bisected to the zstd version update. At that
> point the update can be reverted, and we can work with upstream to find and fix
> the bug. After this big switch in how the kernel consumes zstd, future patches
> will be smaller, because they will only have one upstream release worth of
> changes each.
> 
> This patchset adds a new kernel-style wrapper around zstd. This wrapper API is
> functionally equivalent to the subset of the current zstd API that is currently
> used. The wrapper API changes to be kernel style so that the symbols don't
> collide with zstd's symbols. The update to zstd-1.4.6 maintains the same API
> and preserves the semantics, so that none of the callers need to be updated.
> 
> This patchset comes in 2 parts:
> 1. The first 2 patches prepare for the zstd upgrade. The first patch adds the
>   new kernel style API so zstd can be upgraded without modifying any callers.
>   The second patch adds an indirection for the lib/decompress_unzstd.c
>   including of all decompression source files.
> 2. Import zstd-1.4.10. This patch is completely generated from upstream using
>   automated tooling.
> 
> I tested every caller of zstd on x86_64. I tested both after the 1.4.10 upgrade
> using the compatibility wrapper, and after the final patch in this series.
> 
> I tested kernel and initramfs decompression in i386 and arm.
> 
> I ran benchmarks to compare the current zstd in the kernel with zstd-1.4.6.
> I benchmarked on x86_64 using QEMU with KVM enabled on an Intel i9-9900k.
> I found:
> * BtrFS zstd compression at levels 1 and 3 is 5% faster
> * BtrFS zstd decompression+read is 15% faster
> * SquashFS zstd decompression+read is 15% faster
> * F2FS zstd compression+write at level 3 is 8% faster
> * F2FS zstd decompression+read is 20% faster
> * ZRAM decompression+read is 30% faster
> * Kernel zstd decompression is 35% faster
> * Initramfs zstd decompression+build is 5% faster
> 
> The latest zstd also offers bug fixes. For example the problem with large kernel
> decompression has been fixed upstream for over 2 years
> https://lkml.org/lkml/2020/9/29/27.
> 
> Please let me know if there is anything that I can do to ease the way for these
> patches. I think it is important because it gets large performance improvements,
> contains bug fixes, and is switching to a more maintainable model of consuming
> upstream zstd directly, making it easy to keep up to date.
> 
> Best,
> Nick Terrell
> 
> v1 -> v2:
> * Successfully tested F2FS with help from Chao Yu to fix my test.
> * (1/9) Fix ZSTD_initCStream() wrapper to handle pledged_src_size=0 means unknown.
>  This fixes F2FS with the zstd-1.4.6 compatibility wrapper, exposed by the test.
> 
> v2 -> v3:
> * (3/9) Silence warnings by Kernel Test Robot:
>  https://github.com/facebook/zstd/pull/2324
>  Stack size warnings remain, but these aren't new, and the functions it warns on
>  are either unused or not in the maximum stack path. This patchset reduces zstd
>  compression stack usage by 1 KB overall. I've gotten the low hanging fruit, and
>  more stack reduction would require significant changes that have the potential
>  to introduce new bugs. However, I do hope to continue to reduce zstd stack
>  usage in future versions.
> 
> v3 -> v4:
> * (3/9) Fix errors and warnings reported by Kernel Test Robot:
>  https://github.com/facebook/zstd/pull/2326
>  - Replace mem.h with a custom kernel implementation that matches the current
>    lib/zstd/mem.h in the kernel. This avoids calls to __builtin_bswap*() which
>    don't work on certain architectures, as exposed by the Kernel Test Robot.
>  - Remove ASAN/MSAN (un)poisoning code which doesn't work in the kernel, as
>    exposed by the Kernel Test Robot.
>  - I've fixed all of the valid cppcheck warnings reported, but there were many
>    false positives, where cppcheck was incorrectly analyzing the situation,
>    which I did not fix. I don't believe it is reasonable to expect that upstream
>    zstd silences all the static analyzer false positives. Upstream zstd uses
>    clang scan-build for its static analysis. We find that supporting multiple
>    static analysis tools multiplies the burden of silencing false positives,
>    without providing enough marginal value over running a single static analysis
>    tool.
> 
> v4 -> v5:
> * Rebase onto v5.10-rc2
> * (6/9) Merge with other F2FS changes (no functional change in patch).
> 
> v5 -> v6:
> * Rebase onto v5.10-rc6.
> * Switch to using a kernel style wrapper API as suggested by Cristoph.
> 
> v6 -> v7:
> * Expose the upstream library header as `include/linux/zstd_lib.h`.
>  Instead of creating new structs mirroring the upstream zstd structs
>  use upstream's structs directly with a typedef to get a kernel style name.
>  This removes the memcpy cruft.
> * (1/3) Undo ZSTD_WINDOWLOG_MAX and handle_zstd_error changes.
> * (3/3) Expose zstd_errors.h as `include/linux/zstd_errors.h` because it
>  is needed by the kernel wrapper API.
> 
> v7 -> v8:
> * (1/3) Fix typo in EXPORT_SYMBOL().
> * (1/3) Fix typo in zstd.h comments.
> * (3/3) Update to latest zstd release: 1.4.6 -> 1.4.10
>        This includes ~1KB of stack space reductions.
> 
> v8 -> v9:
> * (1/3) Rebase onto v5.12-rc5
> * (1/3) Add zstd_min_clevel() & zstd_max_clevel() and use in f2fs.
>        Thanks to Oleksandr Natalenko for spotting it!
> * (1/3) Move lib/decompress_unzstd.c usage of ZSTD_getErrorCode()
>        to zstd_get_error_code().
> * (1/3) Update modified zstd headers to yearless copyright.
> * (2/3) Add copyright/license header to decompress_sources.h for consistency.
> * (3/3) Update to yearless copyright for all zstd files. Thanks to
>        Mike Dolan for spotting it!
> 
> v9 -> v10:
> * Add a 4th patch in the series which adds an entry for zstd to MAINTAINERS.
> 
> v10 -> v11:
> * Rebase cleanly onto v5.12-rc8
> * (3/4) Replace invalid kernel style comments in zstd with regular comments.
>  Thanks to Randy Dunlap for the suggestion.
> 
> Nick Terrell (4):
>  lib: zstd: Add kernel-specific API
>  lib: zstd: Add decompress_sources.h for decompress_unzstd
>  lib: zstd: Upgrade to latest upstream zstd version 1.4.10
>  MAINTAINERS: Add maintainer entry for zstd
> 
> MAINTAINERS                                   |   12 +
> crypto/zstd.c                                 |   28 +-
> fs/btrfs/zstd.c                               |   68 +-
> fs/f2fs/compress.c                            |   56 +-
> fs/f2fs/super.c                               |    2 +-
> fs/pstore/platform.c                          |    2 +-
> fs/squashfs/zstd_wrapper.c                    |   16 +-
> include/linux/zstd.h                          | 1252 +---
> include/linux/zstd_errors.h                   |   77 +
> include/linux/zstd_lib.h                      | 2432 ++++++++
> lib/decompress_unzstd.c                       |   48 +-
> lib/zstd/Makefile                             |   44 +-
> lib/zstd/bitstream.h                          |  380 --
> lib/zstd/common/bitstream.h                   |  437 ++
> lib/zstd/common/compiler.h                    |  151 +
> lib/zstd/common/cpu.h                         |  194 +
> lib/zstd/common/debug.c                       |   24 +
> lib/zstd/common/debug.h                       |  101 +
> lib/zstd/common/entropy_common.c              |  357 ++
> lib/zstd/common/error_private.c               |   56 +
> lib/zstd/common/error_private.h               |   66 +
> lib/zstd/common/fse.h                         |  710 +++
> lib/zstd/common/fse_decompress.c              |  390 ++
> lib/zstd/common/huf.h                         |  356 ++
> lib/zstd/common/mem.h                         |  259 +
> lib/zstd/common/zstd_common.c                 |   83 +
> lib/zstd/common/zstd_deps.h                   |  125 +
> lib/zstd/common/zstd_internal.h               |  450 ++
> lib/zstd/compress.c                           | 3485 -----------
> lib/zstd/compress/fse_compress.c              |  625 ++
> lib/zstd/compress/hist.c                      |  165 +
> lib/zstd/compress/hist.h                      |   75 +
> lib/zstd/compress/huf_compress.c              |  902 +++
> lib/zstd/compress/zstd_compress.c             | 5105 +++++++++++++++++
> lib/zstd/compress/zstd_compress_internal.h    | 1188 ++++
> lib/zstd/compress/zstd_compress_literals.c    |  158 +
> lib/zstd/compress/zstd_compress_literals.h    |   29 +
> lib/zstd/compress/zstd_compress_sequences.c   |  439 ++
> lib/zstd/compress/zstd_compress_sequences.h   |   54 +
> lib/zstd/compress/zstd_compress_superblock.c  |  850 +++
> lib/zstd/compress/zstd_compress_superblock.h  |   32 +
> lib/zstd/compress/zstd_cwksp.h                |  482 ++
> lib/zstd/compress/zstd_double_fast.c          |  521 ++
> lib/zstd/compress/zstd_double_fast.h          |   32 +
> lib/zstd/compress/zstd_fast.c                 |  496 ++
> lib/zstd/compress/zstd_fast.h                 |   31 +
> lib/zstd/compress/zstd_lazy.c                 | 1412 +++++
> lib/zstd/compress/zstd_lazy.h                 |   81 +
> lib/zstd/compress/zstd_ldm.c                  |  686 +++
> lib/zstd/compress/zstd_ldm.h                  |  110 +
> lib/zstd/compress/zstd_ldm_geartab.h          |  103 +
> lib/zstd/compress/zstd_opt.c                  | 1345 +++++
> lib/zstd/compress/zstd_opt.h                  |   50 +
> lib/zstd/decompress.c                         | 2531 --------
> lib/zstd/decompress/huf_decompress.c          | 1206 ++++
> lib/zstd/decompress/zstd_ddict.c              |  241 +
> lib/zstd/decompress/zstd_ddict.h              |   44 +
> lib/zstd/decompress/zstd_decompress.c         | 2075 +++++++
> lib/zstd/decompress/zstd_decompress_block.c   | 1540 +++++
> lib/zstd/decompress/zstd_decompress_block.h   |   62 +
> .../decompress/zstd_decompress_internal.h     |  202 +
> lib/zstd/decompress_sources.h                 |   28 +
> lib/zstd/entropy_common.c                     |  243 -
> lib/zstd/error_private.h                      |   53 -
> lib/zstd/fse.h                                |  575 --
> lib/zstd/fse_compress.c                       |  795 ---
> lib/zstd/fse_decompress.c                     |  325 --
> lib/zstd/huf.h                                |  212 -
> lib/zstd/huf_compress.c                       |  773 ---
> lib/zstd/huf_decompress.c                     |  960 ----
> lib/zstd/mem.h                                |  151 -
> lib/zstd/zstd_common.c                        |   75 -
> lib/zstd/zstd_compress_module.c               |  124 +
> lib/zstd/zstd_decompress_module.c             |  105 +
> lib/zstd/zstd_internal.h                      |  273 -
> lib/zstd/zstd_opt.h                           | 1014 ----
> 76 files changed, 27299 insertions(+), 12940 deletions(-)
> create mode 100644 include/linux/zstd_errors.h
> create mode 100644 include/linux/zstd_lib.h
> delete mode 100644 lib/zstd/bitstream.h
> create mode 100644 lib/zstd/common/bitstream.h
> create mode 100644 lib/zstd/common/compiler.h
> create mode 100644 lib/zstd/common/cpu.h
> create mode 100644 lib/zstd/common/debug.c
> create mode 100644 lib/zstd/common/debug.h
> create mode 100644 lib/zstd/common/entropy_common.c
> create mode 100644 lib/zstd/common/error_private.c
> create mode 100644 lib/zstd/common/error_private.h
> create mode 100644 lib/zstd/common/fse.h
> create mode 100644 lib/zstd/common/fse_decompress.c
> create mode 100644 lib/zstd/common/huf.h
> create mode 100644 lib/zstd/common/mem.h
> create mode 100644 lib/zstd/common/zstd_common.c
> create mode 100644 lib/zstd/common/zstd_deps.h
> create mode 100644 lib/zstd/common/zstd_internal.h
> delete mode 100644 lib/zstd/compress.c
> create mode 100644 lib/zstd/compress/fse_compress.c
> create mode 100644 lib/zstd/compress/hist.c
> create mode 100644 lib/zstd/compress/hist.h
> create mode 100644 lib/zstd/compress/huf_compress.c
> create mode 100644 lib/zstd/compress/zstd_compress.c
> create mode 100644 lib/zstd/compress/zstd_compress_internal.h
> create mode 100644 lib/zstd/compress/zstd_compress_literals.c
> create mode 100644 lib/zstd/compress/zstd_compress_literals.h
> create mode 100644 lib/zstd/compress/zstd_compress_sequences.c
> create mode 100644 lib/zstd/compress/zstd_compress_sequences.h
> create mode 100644 lib/zstd/compress/zstd_compress_superblock.c
> create mode 100644 lib/zstd/compress/zstd_compress_superblock.h
> create mode 100644 lib/zstd/compress/zstd_cwksp.h
> create mode 100644 lib/zstd/compress/zstd_double_fast.c
> create mode 100644 lib/zstd/compress/zstd_double_fast.h
> create mode 100644 lib/zstd/compress/zstd_fast.c
> create mode 100644 lib/zstd/compress/zstd_fast.h
> create mode 100644 lib/zstd/compress/zstd_lazy.c
> create mode 100644 lib/zstd/compress/zstd_lazy.h
> create mode 100644 lib/zstd/compress/zstd_ldm.c
> create mode 100644 lib/zstd/compress/zstd_ldm.h
> create mode 100644 lib/zstd/compress/zstd_ldm_geartab.h
> create mode 100644 lib/zstd/compress/zstd_opt.c
> create mode 100644 lib/zstd/compress/zstd_opt.h
> delete mode 100644 lib/zstd/decompress.c
> create mode 100644 lib/zstd/decompress/huf_decompress.c
> create mode 100644 lib/zstd/decompress/zstd_ddict.c
> create mode 100644 lib/zstd/decompress/zstd_ddict.h
> create mode 100644 lib/zstd/decompress/zstd_decompress.c
> create mode 100644 lib/zstd/decompress/zstd_decompress_block.c
> create mode 100644 lib/zstd/decompress/zstd_decompress_block.h
> create mode 100644 lib/zstd/decompress/zstd_decompress_internal.h
> create mode 100644 lib/zstd/decompress_sources.h
> delete mode 100644 lib/zstd/entropy_common.c
> delete mode 100644 lib/zstd/error_private.h
> delete mode 100644 lib/zstd/fse.h
> delete mode 100644 lib/zstd/fse_compress.c
> delete mode 100644 lib/zstd/fse_decompress.c
> delete mode 100644 lib/zstd/huf.h
> delete mode 100644 lib/zstd/huf_compress.c
> delete mode 100644 lib/zstd/huf_decompress.c
> delete mode 100644 lib/zstd/mem.h
> delete mode 100644 lib/zstd/zstd_common.c
> create mode 100644 lib/zstd/zstd_compress_module.c
> create mode 100644 lib/zstd/zstd_decompress_module.c
> delete mode 100644 lib/zstd/zstd_internal.h
> delete mode 100644 lib/zstd/zstd_opt.h
> 
> -- 
> 2.31.1
> 

[GIT PULL][PATCH v11 0/4] Update to zstd-1.4.10

[Nick Terrell]

From: Nick Terrell <terrelln@fb.com>

Please pull from

  git@github.com:terrelln/linux.git tags/v11-zstd-1.4.10

to get these changes. Alternatively the patchset is included.

This patchset lists me as the maintainer for zstd and upgrades the zstd library
to the latest upstream release. The current zstd version in the kernel is a
modified version of upstream zstd-1.3.1. At the time it was integrated, zstd
wasn't ready to be used in the kernel as-is. But, it is now possible to use
upstream zstd directly in the kernel.

I have not yet released zstd-1.4.10 upstream. I want the zstd version in the
kernel to match up with a known upstream release, so we know exactly what code
is running. Whenever this patchset is ready for merge, I will cut a release at
the upstream commit that gets merged. This should not be necessary for future
releases.

The kernel zstd library is automatically generated from upstream zstd. A script
makes the necessary changes and imports it into the kernel. The changes are:

1. Replace all libc dependencies with kernel replacements and rewrite includes.
2. Remove unncessary portability macros like: #if defined(_MSC_VER).
3. Use the kernel xxhash instead of bundling it.

This automation gets tested every commit by upstream's continuous integration.
When we cut a new zstd release, we will submit a patch to the kernel to update
the zstd version in the kernel.

I've updated zstd to upstream with one big patch because every commit must build,
so that precludes partial updates. Since the commit is 100% generated, I hope the
review burden is lightened. I considered replaying upstream commits, but that is
not possible because there have been ~3500 upstream commits since the last zstd
import, and the commits don't all build individually. The bulk update preserves
bisectablity because bugs can be bisected to the zstd version update. At that
point the update can be reverted, and we can work with upstream to find and fix
the bug. After this big switch in how the kernel consumes zstd, future patches
will be smaller, because they will only have one upstream release worth of
changes each.

This patchset adds a new kernel-style wrapper around zstd. This wrapper API is
functionally equivalent to the subset of the current zstd API that is currently
used. The wrapper API changes to be kernel style so that the symbols don't
collide with zstd's symbols. The update to zstd-1.4.6 maintains the same API
and preserves the semantics, so that none of the callers need to be updated.

This patchset comes in 2 parts:
1. The first 2 patches prepare for the zstd upgrade. The first patch adds the
   new kernel style API so zstd can be upgraded without modifying any callers.
   The second patch adds an indirection for the lib/decompress_unzstd.c
   including of all decompression source files.
2. Import zstd-1.4.10. This patch is completely generated from upstream using
   automated tooling.

I tested every caller of zstd on x86_64. I tested both after the 1.4.10 upgrade
using the compatibility wrapper, and after the final patch in this series.

I tested kernel and initramfs decompression in i386 and arm.

I ran benchmarks to compare the current zstd in the kernel with zstd-1.4.6.
I benchmarked on x86_64 using QEMU with KVM enabled on an Intel i9-9900k.
I found:
* BtrFS zstd compression at levels 1 and 3 is 5% faster
* BtrFS zstd decompression+read is 15% faster
* SquashFS zstd decompression+read is 15% faster
* F2FS zstd compression+write at level 3 is 8% faster
* F2FS zstd decompression+read is 20% faster
* ZRAM decompression+read is 30% faster
* Kernel zstd decompression is 35% faster
* Initramfs zstd decompression+build is 5% faster

The latest zstd also offers bug fixes. For example the problem with large kernel
decompression has been fixed upstream for over 2 years
https://lkml.org/lkml/2020/9/29/27.

Please let me know if there is anything that I can do to ease the way for these
patches. I think it is important because it gets large performance improvements,
contains bug fixes, and is switching to a more maintainable model of consuming
upstream zstd directly, making it easy to keep up to date.

Best,
Nick Terrell

v1 -> v2:
* Successfully tested F2FS with help from Chao Yu to fix my test.
* (1/9) Fix ZSTD_initCStream() wrapper to handle pledged_src_size=0 means unknown.
  This fixes F2FS with the zstd-1.4.6 compatibility wrapper, exposed by the test.

v2 -> v3:
* (3/9) Silence warnings by Kernel Test Robot:
  https://github.com/facebook/zstd/pull/2324
  Stack size warnings remain, but these aren't new, and the functions it warns on
  are either unused or not in the maximum stack path. This patchset reduces zstd
  compression stack usage by 1 KB overall. I've gotten the low hanging fruit, and
  more stack reduction would require significant changes that have the potential
  to introduce new bugs. However, I do hope to continue to reduce zstd stack
  usage in future versions.

v3 -> v4:
* (3/9) Fix errors and warnings reported by Kernel Test Robot:
  https://github.com/facebook/zstd/pull/2326
  - Replace mem.h with a custom kernel implementation that matches the current
    lib/zstd/mem.h in the kernel. This avoids calls to __builtin_bswap*() which
    don't work on certain architectures, as exposed by the Kernel Test Robot.
  - Remove ASAN/MSAN (un)poisoning code which doesn't work in the kernel, as
    exposed by the Kernel Test Robot.
  - I've fixed all of the valid cppcheck warnings reported, but there were many
    false positives, where cppcheck was incorrectly analyzing the situation,
    which I did not fix. I don't believe it is reasonable to expect that upstream
    zstd silences all the static analyzer false positives. Upstream zstd uses
    clang scan-build for its static analysis. We find that supporting multiple
    static analysis tools multiplies the burden of silencing false positives,
    without providing enough marginal value over running a single static analysis
    tool.

v4 -> v5:
* Rebase onto v5.10-rc2
* (6/9) Merge with other F2FS changes (no functional change in patch).

v5 -> v6:
* Rebase onto v5.10-rc6.
* Switch to using a kernel style wrapper API as suggested by Cristoph.

v6 -> v7:
* Expose the upstream library header as `include/linux/zstd_lib.h`.
  Instead of creating new structs mirroring the upstream zstd structs
  use upstream's structs directly with a typedef to get a kernel style name.
  This removes the memcpy cruft.
* (1/3) Undo ZSTD_WINDOWLOG_MAX and handle_zstd_error changes.
* (3/3) Expose zstd_errors.h as `include/linux/zstd_errors.h` because it
  is needed by the kernel wrapper API.

v7 -> v8:
* (1/3) Fix typo in EXPORT_SYMBOL().
* (1/3) Fix typo in zstd.h comments.
* (3/3) Update to latest zstd release: 1.4.6 -> 1.4.10
        This includes ~1KB of stack space reductions.

v8 -> v9:
* (1/3) Rebase onto v5.12-rc5
* (1/3) Add zstd_min_clevel() & zstd_max_clevel() and use in f2fs.
        Thanks to Oleksandr Natalenko for spotting it!
* (1/3) Move lib/decompress_unzstd.c usage of ZSTD_getErrorCode()
        to zstd_get_error_code().
* (1/3) Update modified zstd headers to yearless copyright.
* (2/3) Add copyright/license header to decompress_sources.h for consistency.
* (3/3) Update to yearless copyright for all zstd files. Thanks to
        Mike Dolan for spotting it!

v9 -> v10:
* Add a 4th patch in the series which adds an entry for zstd to MAINTAINERS.

v10 -> v11:
* Rebase cleanly onto v5.12-rc8
* (3/4) Replace invalid kernel style comments in zstd with regular comments.
  Thanks to Randy Dunlap for the suggestion.

Nick Terrell (4):
  lib: zstd: Add kernel-specific API
  lib: zstd: Add decompress_sources.h for decompress_unzstd
  lib: zstd: Upgrade to latest upstream zstd version 1.4.10
  MAINTAINERS: Add maintainer entry for zstd

 MAINTAINERS                                   |   12 +
 crypto/zstd.c                                 |   28 +-
 fs/btrfs/zstd.c                               |   68 +-
 fs/f2fs/compress.c                            |   56 +-
 fs/f2fs/super.c                               |    2 +-
 fs/pstore/platform.c                          |    2 +-
 fs/squashfs/zstd_wrapper.c                    |   16 +-
 include/linux/zstd.h                          | 1252 +---
 include/linux/zstd_errors.h                   |   77 +
 include/linux/zstd_lib.h                      | 2432 ++++++++
 lib/decompress_unzstd.c                       |   48 +-
 lib/zstd/Makefile                             |   44 +-
 lib/zstd/bitstream.h                          |  380 --
 lib/zstd/common/bitstream.h                   |  437 ++
 lib/zstd/common/compiler.h                    |  151 +
 lib/zstd/common/cpu.h                         |  194 +
 lib/zstd/common/debug.c                       |   24 +
 lib/zstd/common/debug.h                       |  101 +
 lib/zstd/common/entropy_common.c              |  357 ++
 lib/zstd/common/error_private.c               |   56 +
 lib/zstd/common/error_private.h               |   66 +
 lib/zstd/common/fse.h                         |  710 +++
 lib/zstd/common/fse_decompress.c              |  390 ++
 lib/zstd/common/huf.h                         |  356 ++
 lib/zstd/common/mem.h                         |  259 +
 lib/zstd/common/zstd_common.c                 |   83 +
 lib/zstd/common/zstd_deps.h                   |  125 +
 lib/zstd/common/zstd_internal.h               |  450 ++
 lib/zstd/compress.c                           | 3485 -----------
 lib/zstd/compress/fse_compress.c              |  625 ++
 lib/zstd/compress/hist.c                      |  165 +
 lib/zstd/compress/hist.h                      |   75 +
 lib/zstd/compress/huf_compress.c              |  902 +++
 lib/zstd/compress/zstd_compress.c             | 5105 +++++++++++++++++
 lib/zstd/compress/zstd_compress_internal.h    | 1188 ++++
 lib/zstd/compress/zstd_compress_literals.c    |  158 +
 lib/zstd/compress/zstd_compress_literals.h    |   29 +
 lib/zstd/compress/zstd_compress_sequences.c   |  439 ++
 lib/zstd/compress/zstd_compress_sequences.h   |   54 +
 lib/zstd/compress/zstd_compress_superblock.c  |  850 +++
 lib/zstd/compress/zstd_compress_superblock.h  |   32 +
 lib/zstd/compress/zstd_cwksp.h                |  482 ++
 lib/zstd/compress/zstd_double_fast.c          |  521 ++
 lib/zstd/compress/zstd_double_fast.h          |   32 +
 lib/zstd/compress/zstd_fast.c                 |  496 ++
 lib/zstd/compress/zstd_fast.h                 |   31 +
 lib/zstd/compress/zstd_lazy.c                 | 1412 +++++
 lib/zstd/compress/zstd_lazy.h                 |   81 +
 lib/zstd/compress/zstd_ldm.c                  |  686 +++
 lib/zstd/compress/zstd_ldm.h                  |  110 +
 lib/zstd/compress/zstd_ldm_geartab.h          |  103 +
 lib/zstd/compress/zstd_opt.c                  | 1345 +++++
 lib/zstd/compress/zstd_opt.h                  |   50 +
 lib/zstd/decompress.c                         | 2531 --------
 lib/zstd/decompress/huf_decompress.c          | 1206 ++++
 lib/zstd/decompress/zstd_ddict.c              |  241 +
 lib/zstd/decompress/zstd_ddict.h              |   44 +
 lib/zstd/decompress/zstd_decompress.c         | 2075 +++++++
 lib/zstd/decompress/zstd_decompress_block.c   | 1540 +++++
 lib/zstd/decompress/zstd_decompress_block.h   |   62 +
 .../decompress/zstd_decompress_internal.h     |  202 +
 lib/zstd/decompress_sources.h                 |   28 +
 lib/zstd/entropy_common.c                     |  243 -
 lib/zstd/error_private.h                      |   53 -
 lib/zstd/fse.h                                |  575 --
 lib/zstd/fse_compress.c                       |  795 ---
 lib/zstd/fse_decompress.c                     |  325 --
 lib/zstd/huf.h                                |  212 -
 lib/zstd/huf_compress.c                       |  773 ---
 lib/zstd/huf_decompress.c                     |  960 ----
 lib/zstd/mem.h                                |  151 -
 lib/zstd/zstd_common.c                        |   75 -
 lib/zstd/zstd_compress_module.c               |  124 +
 lib/zstd/zstd_decompress_module.c             |  105 +
 lib/zstd/zstd_internal.h                      |  273 -
 lib/zstd/zstd_opt.h                           | 1014 ----
 76 files changed, 27299 insertions(+), 12940 deletions(-)
 create mode 100644 include/linux/zstd_errors.h
 create mode 100644 include/linux/zstd_lib.h
 delete mode 100644 lib/zstd/bitstream.h
 create mode 100644 lib/zstd/common/bitstream.h
 create mode 100644 lib/zstd/common/compiler.h
 create mode 100644 lib/zstd/common/cpu.h
 create mode 100644 lib/zstd/common/debug.c
 create mode 100644 lib/zstd/common/debug.h
 create mode 100644 lib/zstd/common/entropy_common.c
 create mode 100644 lib/zstd/common/error_private.c
 create mode 100644 lib/zstd/common/error_private.h
 create mode 100644 lib/zstd/common/fse.h
 create mode 100644 lib/zstd/common/fse_decompress.c
 create mode 100644 lib/zstd/common/huf.h
 create mode 100644 lib/zstd/common/mem.h
 create mode 100644 lib/zstd/common/zstd_common.c
 create mode 100644 lib/zstd/common/zstd_deps.h
 create mode 100644 lib/zstd/common/zstd_internal.h
 delete mode 100644 lib/zstd/compress.c
 create mode 100644 lib/zstd/compress/fse_compress.c
 create mode 100644 lib/zstd/compress/hist.c
 create mode 100644 lib/zstd/compress/hist.h
 create mode 100644 lib/zstd/compress/huf_compress.c
 create mode 100644 lib/zstd/compress/zstd_compress.c
 create mode 100644 lib/zstd/compress/zstd_compress_internal.h
 create mode 100644 lib/zstd/compress/zstd_compress_literals.c
 create mode 100644 lib/zstd/compress/zstd_compress_literals.h
 create mode 100644 lib/zstd/compress/zstd_compress_sequences.c
 create mode 100644 lib/zstd/compress/zstd_compress_sequences.h
 create mode 100644 lib/zstd/compress/zstd_compress_superblock.c
 create mode 100644 lib/zstd/compress/zstd_compress_superblock.h
 create mode 100644 lib/zstd/compress/zstd_cwksp.h
 create mode 100644 lib/zstd/compress/zstd_double_fast.c
 create mode 100644 lib/zstd/compress/zstd_double_fast.h
 create mode 100644 lib/zstd/compress/zstd_fast.c
 create mode 100644 lib/zstd/compress/zstd_fast.h
 create mode 100644 lib/zstd/compress/zstd_lazy.c
 create mode 100644 lib/zstd/compress/zstd_lazy.h
 create mode 100644 lib/zstd/compress/zstd_ldm.c
 create mode 100644 lib/zstd/compress/zstd_ldm.h
 create mode 100644 lib/zstd/compress/zstd_ldm_geartab.h
 create mode 100644 lib/zstd/compress/zstd_opt.c
 create mode 100644 lib/zstd/compress/zstd_opt.h
 delete mode 100644 lib/zstd/decompress.c
 create mode 100644 lib/zstd/decompress/huf_decompress.c
 create mode 100644 lib/zstd/decompress/zstd_ddict.c
 create mode 100644 lib/zstd/decompress/zstd_ddict.h
 create mode 100644 lib/zstd/decompress/zstd_decompress.c
 create mode 100644 lib/zstd/decompress/zstd_decompress_block.c
 create mode 100644 lib/zstd/decompress/zstd_decompress_block.h
 create mode 100644 lib/zstd/decompress/zstd_decompress_internal.h
 create mode 100644 lib/zstd/decompress_sources.h
 delete mode 100644 lib/zstd/entropy_common.c
 delete mode 100644 lib/zstd/error_private.h
 delete mode 100644 lib/zstd/fse.h
 delete mode 100644 lib/zstd/fse_compress.c
 delete mode 100644 lib/zstd/fse_decompress.c
 delete mode 100644 lib/zstd/huf.h
 delete mode 100644 lib/zstd/huf_compress.c
 delete mode 100644 lib/zstd/huf_decompress.c
 delete mode 100644 lib/zstd/mem.h
 delete mode 100644 lib/zstd/zstd_common.c
 create mode 100644 lib/zstd/zstd_compress_module.c
 create mode 100644 lib/zstd/zstd_decompress_module.c
 delete mode 100644 lib/zstd/zstd_internal.h
 delete mode 100644 lib/zstd/zstd_opt.h

-- 
2.31.1